Skip to content

Voice chat#

You can talk to Prism Agent out loud. Your client sends your microphone, the agent answers by voice, and its tools run while you speak. Where voice meetings put the agent in a Google Meet call, a voice chat is a direct conversation between the agent and one client.

Nothing about a call is written down. There is no transcript and no message appears in the thread, so what you hear is all the call produces.

Requirements#

A voice chat needs a Realtime credential: managed OpenAI through Lens Agents, an explicit OPENAI_REALTIME_URL and OPENAI_REALTIME_API_KEY, or OPENAI_API_KEY direct to api.openai.com. See Realtime endpoint and credential for the resolution order, and Voice meetings for the realtime model and voice variables.

An egress-filtered deployment must allow the host that resolution reaches. A runtime that cannot reach it refuses the call.

The agent has one voice, so it holds one call at a time. A voice chat is refused while a Meet call is live, and a Meet call is refused while a voice chat is live.

Starting a call#

Press the microphone in the chat composer of the web UI. The browser asks for the microphone the first time, and the same button ends the call. While a call runs, a mute button beside it closes the microphone: the agent hears nothing until you press it again, and keeps talking to you in the meantime.

The terminal UI has no voice control.

The agent's desktop is not a call of its own — see The agent's computer, where Talk starts a voice chat like this one beside the screen.

Who the agent answers#

A call is either a conversation or a room, and the client says which when the call starts. Changing it means ending the call and starting another.

  • A conversation is the default. One person is talking to the agent, so everything the microphone hears is meant for it, and the agent answers each utterance as soon as you stop speaking.
  • A room is several people talking, most of it to each other. The agent listens to all of it and answers only the utterances aimed at it, so somebody has to call it by name. A name that is mentioned rather than used — "I asked Prism about that yesterday" — does not summon it, and a follow-up inside an exchange it is already part of reaches it without the name being repeated. Expect long silences, which is the agent working, and answers that start about a second later than in a conversation.

What the agent will do#

The agent keeps every tool it has in a call except file sharing. Ask the agent for the file in chat afterwards.

Nothing else is held back, so the guardrail is an instruction in the agent's prompt rather than the tool list: it announces what it is about to do and gets an explicit "yes" before anything that deletes, deploys, spends money, grants access, or changes its own configuration.

That includes changing itself — renaming itself, or rewriting its Soul, user profile, memory, vision, or heartbeat config. A sentence agreed in a call can change the agent for every future turn on every channel, not just for the length of the call. An instruction is a strong default, not enforcement, and waiting to be addressed narrows what the agent responds to rather than what it can be asked to do. For a real restriction, start the agent with a narrower tool surface.

Troubleshooting#

The call is refused. The message says why: a Meet call holds the agent's voice, another client already has the session, or the runtime is still loading its agent and the call can be retried.

The call ends on its own. The agent's model session closed. Start another.

The agent talks over you, or answers nothing. Both are the client's audio handling. It has to send the microphone continuously, silence included, because the silence is how the end of an utterance is found, and it has to drop unplayed audio when the runtime tells it to.

Speech sounds broken on a mobile link. The audio rides the chat connection, which has no jitter buffer and no packet-loss concealment, so a poor link sounds worse here than a media transport would.