The Prompt Said Announce It. The Model Didn't. So the Worker Did: Closing a 15-Second Silence in Déblo Voice

By Juste Thales Gnimavo (CEO, ZeroSuite) and Claude Opus 5.5 (Claude Code)

On October 12, 2026, two bug reports landed in our inbox, both tagged [VOICE BUG HIGH], both filed by Juste from inside a Déblo Voice call. The description was short: the assistant starts an extended search without saying anything, then nothing happens for about 15 seconds. On a phone, 15 seconds of dead air is not a wait. It is a frozen app, and the user hangs up.

Déblo Voice is the real-time voice side of Déblo: a Gemini Live native-audio session, orchestrated by a LiveKit Agents worker that we run ourselves. When a question is hard (an integral, a loan schedule, a multi-step reasoning task), the live model calls a function tool named deep_think, which sends the question to a stronger text model and returns the answer for the voice model to speak. That round trip is where the silence lived.

This post is the account of the fix that shipped on October 15 and was proven in production on October 16. The short version: we stopped asking the model to fill the gap and made the worker do it, deterministically.

The prompt already said it

The first thing we checked was the obvious one. Had we told the model to announce the search before calling the tool? We had. The deep_think tool description carries a block literally titled MANDATORY pre-call narration.

It does not work, and we already had the measurement explaining why. In a smoke test back in May we had logged ttft=-1 on these turns: the native-audio model emits the function_call as its very first token. There is no audio before it. The model is not ignoring the instruction out of carelessness; in that position it simply does not speak before acting. Making the instruction louder once more would change nothing, and we decided not to try.

We also checked whether the API side had anything left to give. The tool already runs as NON_BLOCKING with INTERRUPT scheduling. That lever was used up.

Two silences, not one

The second finding mattered more. Even if the model had announced itself, the prompt route could never have closed the gap, because the silence has two parts.

From the May measurements (on Gemini 3.5 Flash at the time):

1. The tool itself runs: 4 to 7 seconds of execution.
2. Then the live model takes the result and needs time before it starts speaking the reply: a measured post_gap of 7.18 seconds.

An announcement spoken before the call covers the start of part one. It does nothing for the rest of part one, and nothing for part two. Any fix that lives in the prompt is a fix for the first second or two of a 15-second problem.

So the requirements wrote themselves. Something has to speak when the model will not, and something has to keep the line audibly alive until the reply actually starts. Both have to come from code we control.

The fix: two cues, both in the worker

Cue 1: a pre-recorded announcement, in the session's own voice

LiveKit Agents 1.8.5 ships RunContext.with_filler, which runs a callback once the session has been idle for a given delay while a tool is executing. We wrap the deep_think call in it:

pythonif TOOL_FILLER_ENABLED:
    async with run_ctx.with_filler(
        _deep_think_filler, delay=DEEP_THINK_FILLER_DELAY_S,
    ):
        result = await call_deep_think(
            user_id=user_id, question=question, context=context,
        )

The filler plays a pre-recorded clip after 0.4 seconds of idle. The clip has to sound like Déblo, not like a second assistant interrupting. We generated six short clips with Gemini 3.1 Flash TTS, using the same prebuilt voices as the live sessions: Charon for K-12, Enceladus for Pro and Companion, Orus for the personal assistant, each in French and English. They are Opus files between 4.3 and 5.2 seconds long, decoded at 24 kHz. Generating them cost a few cents.

The French text is "Je vérifie ça avec mes ressources étendues, j'ai la réponse dans quelques secondes." The English one: "Let me check that with my extended resources, I'll have your answer in a few seconds."

Three rules govern when it plays:

• Skipped if the model already spoke since the end of the user's last utterance. Sometimes the model does announce itself, or a previous tool in the same chain was already announced. The end of the user's voice comes from the server-side VAD, with the final transcript as a fallback.
• No clip, no announcement. A voice or language without a matching clip gets nothing (only cue 2). Never a wrong voice, never a wrong language.
• Spoken only. The announcement is played with add_to_chat_ctx=False, so it never enters the Gemini context or the saved conversation transcript.

The clip is passed to session.say() as pre-decoded audio frames, so no TTS runs at call time.

Cue 2: a soft typing sound while the agent thinks

The announcement covers about five seconds. The rest of the wait needed something else. We added a quiet keyboard-typing loop on a separate audio track (BackgroundAudioPlayer), active while the agent is in the thinking state.

Before relying on that state, we read the SDK source for 1.8.5: on the realtime path, the agent enters thinking only while a function tool runs and until the model starts speaking the reply. It never enters it on an ordinary conversational turn. That made the state a reliable trigger for every function tool, not only deep_think.

The details that took the most care:

• Muted while the user talks. The SDK stays in thinking during a barge-in as long as a tool is still running. Typing over someone's voice is noise, so the sound stops when the user starts speaking and resumes when they stop.
• One-second fade-in. Fast tools finish before the sound becomes audible, so they stay silent.
• Hard cap at 30 seconds. If the state ever gets stuck in thinking, the loop stops and logs it.

We chose typing over hold music. It is more discreet, and it reads as "working on it" rather than "you are on hold".

No client release

Neither cue required a new app version. The web client already attaches every remote audio track, and the mobile app routes them natively. Both cues ride on tracks the clients were already playing. Three environment variables let us turn each cue off or change the volume without redeploying code.

The trap we almost shipped with

Before pushing, we checked which SDK version was actually running in production, by reading pip freeze inside the worker container. Answer: livekit-agents 1.8.5, image built on October 7.

Then we looked at requirements.txt. It said livekit-agents>=1.5.0. Every rebuild on our hosting platform resolved to the latest release. Version 1.8.8 had been published that same day. Our voice fix, written and verified against 1.8.5, would have shipped together with an untested SDK jump, and the worker monkey-patches SDK internals. If anything had broken, we would have blamed the fix.

We pinned the six LiveKit and Google packages to the exact production versions, and added a constraints.txt with all 89 packages from the production freeze, applied by the Dockerfile. A dry-run resolution for Linux x86_64 produced exactly those 89 packages and no others. The push changed no dependency.

The audit, and what it changed

Before the push, an independent read-only audit agent reviewed the change against the real 1.8.5 SDK source. Verdict: GO-WITH-FIXES, with no deadlock and no crash path. The fixes we applied:

• the typing sound now stops while the user speaks;
• the "model already spoke" check uses the VAD end of voice instead of the final transcript, which can land after the model has already answered;
• event listeners are unsubscribed before the background player is closed;
• say() is wrapped, and the played event is emitted only when a handle exists;
• the constraints.txt above.

We rejected two of its suggestions, and the reasons are worth keeping:

• Choosing the announcement language from the transcript. Our transcript language detector returns English for any French typed or transcribed without accents ("quelle est la derivee"). It would have produced more wrong-language announcements than it prevented. The session language stays the source.
• Raising the idle delay from 0.4 to 0.8 seconds. Every tenth of a second added is pure silence on the common case (the model says nothing) in exchange for reducing a rare one (a duplicate announcement). We kept 0.4.

We also turned down an idea from the bug report itself: asking the user for confirmation before each extended search. It removes the surprise, but it adds a full voice round trip to every complex question. That costs more than the silence we were fixing.

Proof in production

On October 16 at 08:20 UTC, Juste made a call from the mobile app: Pro audience, Enceladus voice, French. He asked two questions that trigger deep_think: the integral of x² eˣ from 0 to 1, and the monthly payment on a 25 million FCFA loan at 9 % over 5 years. He sent us the worker logs. Times below are relative to the tool_call event:

The relevant events for the first call, reconstructed and abridged, with identifiers removed:

textsession.tool_call      tool=deep_think
session.tool_pre_speech tool=deep_think  no_pre_speech=true
session.tool_filler    tool=deep_think  action=played  voice=Enceladus  lang=fr
session.tool_success   tool=deep_think  model=gemini-3.8-flash

On the first call the model hit no_pre_speech: true: it called the tool without saying a word, which is the exact case from the bug report. The announcement played. There was no thinking_sound_error, no thinking_sound_capped, no filler skipped or in error.

Juste's own words, translated: "I heard the audio in the background, like someone typing on a keyboard, and I didn't even notice there was a wait."

The answers were right, too: e − 2 for the integral, and a monthly payment of 518,959 FCFA (exact value 518,958.88). The total interest it announced, 6,137,634 FCFA against 6,137,533 exact, was off by 101 FCFA, a rounding by the model.

One honest note on that table: the model-side silence is now longer than in May, not shorter. Production runs deep_think on gemini-3.8-flash, which takes 9.5 to 12.5 seconds to execute (against 4 to 7 on 3.5), though the live model resumes faster (4.6 to 4.9 seconds, against 7). On October 16 Juste chose to keep 3.8: correctness before latency. That trade became acceptable because of the cues. A 17-second wait you can hear being worked on is a different product from a 17-second wait in silence.

What is still open

We marked the fix as proven on mobile, not everywhere:

• Web proof is still due. The browser client attaches every remote audio track, so the risk is low, but we have not observed it yet.
• Echo on loud speakers. We still need to confirm on a real device that the typing sound cannot trigger Gemini's speech detection through the phone's echo cancellation.
• Late transcripts. If the user's final transcript arrives after the model has announced itself, the pre-recorded announcement can double it. Cost: one redundant sentence.
• Fast tools. If the tool answers in under five seconds, the reply waits for the announcement to finish: up to about 1.6 seconds more.
• Our own metric is now wrong. total_silence_s only counts the model's voice, so it reports 14 to 17 seconds on calls where the user heard 0.4. It needs recalibrating.

Takeaways for realtime voice builders

1. Measure where the first token goes before you write prompt instructions about it. If a native-audio model emits the function call first, no instruction will put speech in front of it.
2. Decompose the silence. Ours had a tool part and a reply part. A fix aimed at only one of them looks right in review and fails in the user's ear.
3. Put user-perceived guarantees in code you control. The worker knows when a tool starts, when the session is idle and when the user is talking. It can make promises the model cannot.
4. Use the session's own voice for filler. A pre-recorded clip in the same prebuilt voice sounds like the assistant, not like a recorded message.
5. Pin what production is actually running, then prove the pin. Read the freeze from the container, constrain it, and resolve it dry before you push anything else on top.
6. Reject audit suggestions with a reason, in writing. Two of the auditor's ideas would have made the product worse. Saying why is what keeps them from coming back.