Skip to content

The agent's computer#

The Computer page in the web UI gives the agent a desktop of its own and shows you its screen. You watch it work, take control of the cursor and keyboard, and finish the steps it cannot do itself.

Ask for website or desktop work wherever the agent already runs — in chat, in a scheduled task, or on a heartbeat.

Requirements#

The agent's desktop needs packages the container image installs, so run the published image or build one with npm run docker:build. A runtime without them does not offer the page, and Computer is absent from the sidebar.

Speaking to the agent on the page is a voice chat, which has a Realtime credential requirement of its own.

Watching the agent work#

Open Computer in the sidebar. The page shows the desktop as it is now — a whole desktop, so the agent can use a browser, a terminal, a file manager, or anything else the image installs.

The desktop belongs to the runtime rather than to the page. Work runs whether or not anybody is watching, and it continues after you leave. One invocation uses the desktop at a time and the rest queue behind it.

A status line under the screen says what is happening — Prism is working, Ready, Waiting for you, You have control, or Another viewer has control — and how many invocations are waiting. Beside it, Chat opens the conversation the current work belongs to and Stop ends that work. Sound on plays the desktop's audio and is on when the page opens.

Taking control#

Click the screen to take control of the cursor and keyboard, and press Esc or Return control to hand them back. While you hold control, pasting types your clipboard into whatever has the focus on the desktop.

One viewer has control at a time. While another viewer holds it, you watch and the screen does not answer your input.

Finishing a step for the agent#

When a sign-in, a CAPTCHA or a device approval blocks it, the agent asks you to finish that step instead of failing the work. The request arrives in the conversation the work came from with a link to Computer, and the agent stops using the desktop and waits. Its other tools — connectors, shell, everything that is not the desktop — keep running meanwhile.

To finish the step:

  1. Open Computer from the link in the request.
  2. Click the screen to take control.
  3. Complete the step on the desktop.
  4. Press Esc or I'm done.

The agent reads the screen again and carries on with the original request. Dismiss ends a request the agent should stop waiting on.

The link's address comes from PRISM_PUBLIC_URL, or from the published origin of a Lens-hosted deployment; see Configuration. With neither, the request says to open Computer in the web UI rather than carrying a link.

Talking to the agent#

Press Talk to speak to the agent while you watch. The call is the voice chat of the chat composer on the desktop's own connection: your microphone starts muted, Unmute opens it, and End voice ends the call and leaves the desktop as it is.

The agent has one voice, so Talk is refused while a call from the chat composer or a meeting holds it, and those are refused while a call here is live. Starting a call on a second Computer page moves it to the newer page, which is what a reload does.

Sharing your own screen#

Press Share screen to show the agent your screen, a window, or a tab, the way you would in a video call. Ask the agent to look and it reads what is there. The share rides the voice call, so press Talk first — ending the call stops the share.

"The screen" on its own means the agent's own screen, since that is the one you are both watching. Say "my screen" for yours.

Troubleshooting#

Computer is missing from the sidebar. The runtime has no desktop toolchain. Run the published container image, or build one with npm run docker:build.

The page says the screen cannot be shown. The browser has no video decoder for the stream. The desktop and its sound continue.

Clicking the screen does nothing. Another viewer has control. The status line says so, and the page follows what they do until they hand it back.

The call is refused. Another call holds the agent's voice. The message says which one.

The page says the screen would not start. A program the desktop needs is missing. The message names it.