Skip to content

Models#

An agent's model call is the one request it always makes. Routing it through the platform is what turns spending limits, PII masking, and per-call audit from aspirations into enforcement.

Managed inference#

The platform holds the provider credential and proxies every call. The agent points its SDK at the platform instead of at the provider, and never holds a model API key.

graph LR
    agent[Agent in sandbox] -->|no key| proxy[LLM proxy]
    proxy --> limits{Spending limit}
    proxy --> mask[PII masking]
    proxy --> audit[(Audit trail)]
    proxy --> provider[Bedrock · Foundry · Mantle · OpenAI · OpenRouter]

Enabling it takes two steps: the install configures a provider, and a policy opts in.

name: agent-base
managedInference:
  enabled: true
  provider: bedrock

Omit managedInference and sandboxes under that policy have no model access at all. It is opt-in, not opt-out.

Provider routes#

Agents call the platform on provider-shaped paths, so an unmodified SDK works by changing only its base URL.

All paths below are relative to /v1/projects/{projectId}. The project-scoped form works for every caller, and it is the form a sandbox is given.

Provider Route
Bedrock /llm/bedrock/{region}/{path}
Foundry — Claude /llm/azure/anthropic/{path}
Foundry — GPT /llm/azure/openai/{path}
Bedrock Mantle — Claude /llm/bedrock-mantle/anthropic/{region}/{path}
Bedrock Mantle — GPT /llm/bedrock-mantle/openai/{region}/{path}
OpenAI /llm/openai/{path}
OpenAI — realtime voice /llm/openai/v1/realtime (WebSocket)
OpenRouter — Anthropic format /llm/openrouter/anthropic/{path}
OpenRouter — OpenAI format /llm/openrouter/openai/{path}

OpenRouter's two surfaces are wire formats rather than model families. It normalizes whichever model it routes to into the format that asked for it, so an Anthropic-SDK agent on that backend can drive a Gemini or Llama model and an OpenAI-SDK agent can drive Claude. Sandboxes on it receive both endpoints and the agent's own SDK picks.

A sandbox never builds these paths itself. The platform gives it the project-scoped base URL its SDK reads, so changing that one variable is all an agent needs. See What the sandbox receives.

Ask a running install what it has configured:

curl -fsS -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
  https://agents.example.com/v1/inference/providers

A policy may only name a provider in that list. Anything else has no route behind it.

Provider selection has to agree#

Three places name a provider, and they must match or the agent will not answer:

  1. The install — which providers were configured. See Inference providers.
  2. The policy — managedInference.provider.
  3. The agent — whatever its own configuration or environment tells it to use.

A mismatch is the most common reason a healthy-looking agent stays silent. nexusctl sandbox describe shows the effective policy, which is the fastest way to check the middle one.

The third no longer has to be configured by hand. Every sandbox with managed inference enabled receives LENS_MANAGED_INFERENCE_PROVIDER — and LLM_PROVIDER carrying the same value, under the name a standalone agent already reads — so an agent can be told which backend it was given instead of being pinned to one at build time. LENS_MANAGED_INFERENCE_MODEL carries the resolved model under a provider-neutral name for the backends that have a portable one. See What every backend tells the agent.

This is what keeps a sandbox template correct across a policy edit: the template is authored before anyone picks a provider, so the values are resolved per sandbox rather than baked in.

What proxying buys you#

Metering. Every call is priced and attributed to the org, user, agent, and sandbox that made it. Spending limits are enforced at the proxy — on breach it returns 429 and the sandbox keeps running, losing model access rather than its process and disk.

Masking. PII can be replaced before the prompt leaves your network and restored in the response. Masking is fail-closed by default: if masking fails, the request does not go out. See Privacy and PII controls.

Audit. Actor, project, provider, model, token counts, cost, result, and duration for every call. See Audit trail.

Cost visibility. Usage-cost summaries and timeseries per org, sliceable by actor.

None of this is available for a call the platform never sees.

Agents that bring their own model#

An agent can also hold its own provider key and reach the provider through an allowed domain instead:

allowedDomains:
  - pattern: api.anthropic.com
    verdict: allow
    transport: upstream

The call is still policy-checked and audited at the network level. What you lose is per-call metering, spending limits, and PII masking, because the platform is forwarding bytes rather than reading a model request.

That is a property of routing, not an oversight. Metering happens because a request travels through the platform's inference endpoint — the forward proxy authenticates the connection and pipes it, and does not intercept outbound TLS to meter it covertly. An allowed provider domain is therefore an allowed unmetered spend, and openrouter.ai is the one to watch: one key reaching roughly 60 vendors makes a direct dial there the cheapest way for an agent to spend money nothing counts.

Prefer managed inference wherever the choice exists, and keep provider hosts denied — the default — where metering has to be guaranteed. See Metering follows routing.

Additional model providers are configured to customer requirements. Talk to us.