Skip to content

The agent's computer#

The Computer page in the web UI gives the agent a desktop of its own and shows you its screen. You watch it work, take control of the cursor and keyboard, and finish the steps it cannot do itself.

Ask for website or desktop work wherever the agent already runs — in chat, in a goal, in a scheduled task, or on a heartbeat.

Requirements#

The agent's desktop needs packages the container image installs, so run the published image or build one with npm run docker:build. A runtime without them does not offer the page, and Computer is absent from the sidebar.

Speaking to the agent on the page is a voice chat, which has a Realtime credential requirement of its own.

Watching the agent work#

Open Computer in the sidebar. The page shows the desktop as it is now — a whole desktop, so the agent can use a browser, a terminal, a file manager, or anything else the image installs.

The desktop belongs to the runtime rather than to the page. Work runs whether or not anybody is watching, and it continues after you leave. One invocation uses the desktop at a time and the rest queue behind it.

A status line under the screen says what is happening — Prism is working, Ready, Waiting for you, You have control, You have control · Prism is stopped, Another viewer has control, or Paused · Waiting for hand-back when a viewer who took control left without handing it back — and how many invocations are waiting. Beside it, Chat opens the conversation the current work belongs to, and Stop ends that work. Sound turns the desktop's audio on and off, and is on when the page opens.

Taking control#

Click the screen to take control of the cursor and keyboard, and press Esc or Return control to hand them back. While you hold control, pasting types your clipboard into whatever has the focus on the desktop.

One viewer has control at a time. While another viewer holds it, you watch and the screen does not answer your input.

Finishing a step for the agent#

When a sign-in, a CAPTCHA, or a device approval blocks it, the agent asks you to finish that step instead of failing the work. A request from a conversation, or from a goal started in one, appears in that conversation. A request from a heartbeat, a webhook, or a scheduled task that has no conversation goes to the web chat, Slack, and Microsoft Teams, and you can answer it in any of them. Such a run waits up to 30 minutes for your answer, then finishes without that step.

While a request waits, the agent can keep working on other things. When you take control to finish the step, Prism stops: it makes no model calls and its running commands pause, so it cannot see what you type. When you hand control back, any password fields on the desktop are cleared, and the browser never offers to save a password.

Sign in with the form#

When a website asks only for a username and a password, the request comes with a sign-in form: in the web chat, in the terminal UI, and on the Computer page for anyone who follows the link. The runtime types the credentials into the website for you.

To sign in:

  1. Check the website shown above the fields.
  2. In Username and Password, enter your credentials. A second sign-in step can ask for the password only.
  3. Click Sign in.

Finish the step on the desktop#

In the web chat, the request opens the live desktop in a resizable panel on the right side of the conversation.

To finish the step:

  1. Click the desktop in the panel to take control. Use fullscreen for more room, or open the Computer page from the link in the request.
  2. Complete the step on the desktop. Type a password on the desktop, never in a chat message, and submit the sign-in before you hand control back.
  3. Press Esc or click I'm done. Closing the panel also hands control back.

The agent reads the screen again and carries on with the original request. If it was working on something else, that work finishes first. The conversation shows the request as completed. Dismiss ends a request that the agent should stop waiting on, and Stop cancels the invocation that uses the desktop.

From Slack, Open Computer opens the Computer page with the request beside the screen. The link's address comes from PRISM_PUBLIC_URL, or from the published origin of a Lens-hosted deployment; see Configuration. With neither, the request says to open Computer in the web UI rather than carrying a link.

Talking to the agent#

Press Talk to speak to the agent while you watch. The call is a voice chat on the desktop's own connection: your microphone starts muted, Unmute opens it, and End voice ends the call and leaves the desktop as it is.

The agent has one voice, so Talk is refused while a call from the chat composer or a meeting holds it, and those are refused while a call here is live. Starting a call on a second Computer page moves it to the newer page, which is what a reload does.

Sharing your own screen#

Press Share screen to show the agent your screen, a window, or a tab, the way you would in a video call. Ask the agent to look and it reads what is there. The share rides the voice call, so press Talk first — ending the call stops the share.

"The screen" on its own means the agent's own screen, since that is the one you are both watching. Say "my screen" for yours.

Troubleshooting#

Computer is missing from the sidebar. The runtime has no desktop toolchain. Run the published container image, or build one with npm run docker:build.

The page says the screen cannot be shown. The browser has no video decoder for the stream. The desktop and its sound continue.

Clicking the screen does nothing. Another viewer has control. The status line says so, and the page follows what they do until they hand it back.

The call is refused. Another call holds the agent's voice. The message says which one.

The page says the screen would not start. A program the desktop needs is missing. The message names it.