ONE

Agents That Listen, and Answer Out Loud

A visitor taps once, talks, and the agent talks back. No text box, no app, no third-party account. Here is the whole loop — the speech recognition, the models doing the speaking, why the audio picture moving matters more than it looks, and the honest truth about the pause in the middle.

voice agents speech chat accessibility

There is a moment in every voice interface where the person stops talking and nothing happens. A second passes. They lean in and say “hello?” — and now they are talking to the machine about the machine instead of about the thing they came for.

Most of the work in shipping voice is spent on that second.

You can hear the result for yourself. Try the live demo → /demo/voice — that page is where you talk to it. Reading about a voice interface is close to useless. The point is what it sounds like when it answers.

The loop, one leg at a time

A voice agent is three jobs in a ring: listen, think, speak. Drop any one of them on a third party and you inherit their account system, their pricing, and their outage. So all three run on the page and on our own routes.

Listen. Where the browser has its own speech recognition — Chrome does — we use it. It runs on the device, transcribes continuously while you are still talking, and never uploads a recording. Where the browser does not have it — Safari and Firefox — the page records instead and uploads the audio to our transcription route, which runs OpenAI’s whisper-1 and hands back text. Uploads are capped at 4MB, and both audio routes shed floods at twenty requests per minute per address, because transcription bills by the minute of audio and a voice widget on a public landing page should not be able to become an unbounded invoice.

Think. The transcript goes to the same chat endpoint the typed chat uses. Same agent, same instructions, same tools. This is worth saying plainly: the voice agent is not a separate product with its own brain. It is your existing agent with ears.

Speak. The reply is posted to our speech route, which prefers OpenAI’s gpt-4o-mini-tts and returns an MP3. Six voices: alloy, echo, fable, onyx, nova, shimmer. You pick one per block. If there is no OpenAI key but the workspace has Cloudflare’s AI binding, it falls back to MeloTTS and still speaks, in one generic voice, with the text capped at 500 characters. The fallback is honest about what it is — the capability probe reports zero available voices on that path, so nothing offers six names that do nothing.

One detail that matters more than it reads: while the agent is speaking, recognition is switched off. Without that, the microphone hears the reply, transcribes it, and sends it back as the visitor’s next message. The agent then talks to itself until the credits run out.

Why the picture moves

Every voice block can paint an aura or a bar of levels that move with the audio. It would be easy to file that under decoration. It is not decoration. It is the answer to the question the person is silently asking: is this thing actually hearing me?

A static microphone icon cannot answer that. A lit-up icon cannot either — it only proves the code thinks it is listening. Bands that jump when you speak and settle when you stop are proof at the only level that counts, because they are computed from the live audio track by real frequency analysis. If the bars move, sound is arriving. If you speak and nothing moves, you know something is wrong in the first half-second rather than after a failed reply.

That feedback has a cost, and we would rather state it than have you discover it. On Chrome, with the visualisation on, you get two microphone prompts. The browser’s own recogniser owns the microphone internally and never exposes an audio track to the page, so drawing the levels for the listening half needs a second capture. Setting the visualisation to none reduces it to one prompt. The visualiser code also loads only if you asked for it, so a block with it switched off never downloads the analysis library at all.

The latency reality

No invented benchmarks here, just the shape of the thing, which is what actually determines how it feels.

The listening leg is free on Chrome. Recognition runs locally and finishes as you finish, so there is no round trip between your last word and the agent starting to think. Off Chrome, the pause after you stop talking is real: the page has to stop the recording, upload it, and wait for transcription before anything else can begin.

The speaking leg has one structural cost that no amount of tuning removes today: speech starts after the reply has finished generating, not sentence by sentence. The page waits for the complete turn, sends the whole text for synthesis, then plays it. There is no streaming synthesis. That means the wait a person feels is the model’s full reply time plus one synthesis, and it is why short, spoken-sounding agent instructions produce a dramatically better experience than the same agent tuned for text. An agent that writes four paragraphs is a bad voice agent regardless of how fast anything runs.

The last honest wrinkle is autoplay. A hands-free reply has no tap in its chain, and browsers are within their rights to refuse to play audio in that situation. The block catches the refusal and shows “Tap to hear the reply” rather than sitting in silence pretending to speak. That is also why hands-free mode is off by default.

What needs a key

Speaking needs either an OpenAI key or Cloudflare’s AI binding. Listening off Chrome needs an OpenAI key, with no fallback — we deliberately did not claim Cloudflare’s transcription model, because wiring a capability probe to report something we have not tested would be a lie the page tells the browser. A workspace with only the Cloudflare binding can therefore speak but cannot listen, and that only bites visitors on browsers without built-in recognition.

Worth ruling out one thing: none of this needs a phone number, Twilio, or an ElevenLabs account. Answering real phone calls is a separate product. The in-page agent shares no code with it and needs none of its credentials.

Where it goes

The voice agent is a page block. Drop it on any published page, pick a layout — orb, bar, or panel — set the voice, the greeting, and the language, and publish. It runs on a landing page, a docs page, a pricing page. It uses the agent you already configured.

The full breakdown of options lives on the voice feature page → /features/voice.

But read that second. Go and talk to it first.