Skip to content

Emergency halt#

An autonomous agent that has gone wrong does not stop because someone opened a ticket. A halt is one org-wide switch that stops the platform acting on that organization's behalf, and it is meant to be thrown by whoever notices first.

It removes reach, not state. Nothing is destroyed, so lifting the halt resumes the work.


What a halt refuses#

Surface Under halt
Managed inference Refused for every principal, a human administrator included. An open stream is cut at the next response boundary.
Connectors Refused on every path that would dial an upstream, REST and MCP alike.
Forward proxy No credentials are handed out.
AWS and cluster proxies Stop forwarding.
Kubernetes exec, attach, port-forward The upgrade is refused.
Every running sandbox Receives a deny-all policy.

A sandbox created while the halt is in force gets the same deny-all policy, so a halt cannot be worked around by starting something new. That policy keeps one route: the platform's own host, which is how a sandbox learns the halt was lifted.

What a halt leaves alone#

Sandboxes keep running. They keep their processes, their disks, and their exposed ports. Administration keeps working — you can read the audit trail, edit policies, and lift the halt, because an incident is exactly when those are needed.

Managed inference is the deliberate exception to "administration keeps working". An agent holding an ordinary API token authenticates as a person would, so gating inference on the sandbox alone would leave open the hole the switch exists to close. A human administrator therefore loses managed inference too, for as long as the halt lasts.


Engage it#

In the admin UI the button sits beside New Project and reads Cut network access. It opens a confirmation dialog that names the organization and asks for the reason:

The confirmation dialog for cutting network access, naming the organization, listing what stops, and requiring a reason.

Over REST:

curl -fsS -X POST \
  -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
  -H "Content-Type: application/json" \
  https://agents.example.com/v1/orgs/$ORG_ID/halt \
  -d '{"reason": "Runaway spend: several agents looping on an expensive model"}'

A reason is required, up to 500 characters. It is recorded permanently and shown to every member of the organization — the first thing an incident review asks for.

Engaging requires org administration. Engaging an organization that is already halted is not an error: the response carries the halt in force and sets alreadyHalted, because two administrators reaching for the switch during one incident is expected, not a mistake.

A repeat engage is also the repair. Every engage re-pushes the halt to every connected sandbox, whether or not it wrote a record, so if a push failed, or a replica died between the write and the push, pressing the button again reaches the sandboxes that push missed. A sandbox that was unreachable when the switch was thrown needs no repeat: it receives the halt when it reconnects. The same is true of a lift.

While a halt is in force, every page in that organization carries a banner reading "This organization's network is cut", and the button becomes Restore network access.

The UI says "cut network access" where this page and the API say "halt". They are the same control: the REST surface is /orgs/{orgId}/halt, and the audit trail records org.halt.engage and org.halt.resume, shown as "Emergency halt engaged" and "Emergency halt lifted".

No CLI, and no agent tool

There is no nexusctl command and no MCP tool for this. The switch is a human decision, so it is reachable from the admin UI and the REST API only. An agent cannot halt an organization, and an agent cannot lift one.

Lift it#

curl -fsS -X DELETE \
  -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
  https://agents.example.com/v1/orgs/$ORG_ID/halt

Lifting requires org administration, and lifting an organization that was never halted is not an error. Sandboxes take their own policies back and resume. The halt itself is never deleted — it stays in the history with the time it was lifted and who lifted it.

Read the state and the history#

curl -fsS -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
  https://agents.example.com/v1/orgs/$ORG_ID/halt

Any member of the organization can read this, not only an administrator. Everyone whose agents have stopped needs to see why.

The answer reports halted, the halt in force if there is one, and history — every halt this organization has declared, newest first. Each record names who engaged it, when, and why, and who lifted it and when. Halts are append-only, so the record survives the lift.


How a refusal reads#

A refusal under halt is its own answer rather than "access denied". That matters at three in the morning: an operator who reads "access denied" goes looking at RBAC, which is the wrong place and costs the minutes the halt exists to save. MCP clients are told in JSON-RPC rather than in an HTTP problem document, so an agent reports the halt instead of a protocol error.

Refused inference is recorded in the audit trail with orgHalted in its metadata, so a halt's blast radius is a query rather than a guess.


What happens when the platform cannot read its own state#

A kill switch that a database blip can silently lift is not a control anybody can certify. So halt resolution fails stale, then closed:

  • A successfully read answer is cached for a few seconds. A change to the switch invalidates that cache immediately, so engaging or lifting takes effect in seconds rather than at the end of a timer.
  • Where the state cannot be read at all, the last answer that was read successfully is served for up to one minute. An organization known to be halted therefore stays halted for as long as the outage lasts.
  • Past that minute, the organization answers as halted, so every surface in the table above refuses — a human administrator's managed inference included. A replica that has never managed a successful read refuses from the start, so a pod that boots mid-outage cannot serve the traffic an engaged halt should have stopped.

Each enforcement point fails closed on its own account as well, so a wiring mistake refuses rather than admits. A sandbox reconciles halt state whenever it reconnects, which covers a sandbox that was unreachable when the switch was thrown.

This is the deliberate inverse of a spending limit, which fails open so that a database outage cannot take down every model call at once.


When to use it#

A halt is the blunt instrument. Reach for it when you do not yet know which agent, which project, or which credential is the problem — runaway spend, a credential you believe is compromised, an agent behaving in a way nobody can explain, or an audit finding that has to stop before it is understood.

When you do know, the narrower control is better and leaves everything else working: deny the host in policy, tighten a spending limit, revoke an API token, or stop the one sandbox.

Rehearse it. A switch nobody has thrown in a drill is a switch nobody trusts during an incident.


Incident-response rehearsals and runbooks for your own environment are worth a conversation. Talk to us.