Codegen sandbox isolation

An agent that can write files is useful, and one that can also reach the network can send those files anywhere. The codegen sandbox keeps the two capabilities apart.

It gives an AgentRuntime the full codegen tool surface over MCP, a shell included, in a pod that by default reaches nothing at all, and may be permitted to reach named in-cluster Services and nothing else.

Every sandbox, codegen or browser, gets the same pod baseline and per-call policy, which Sandbox security explains. This page covers what is specific to codegen: why the write primitive lives in a sandbox at all, the shell, the workspace and how work leaves the sandbox.

It is also the page for the life of a claim: how a runtime acquires a sandbox, how the operator identifies it, and how claims queue, are replaced, renewed and reaped. That lifecycle is the same for browser sandboxes.

How the pieces fit

The runtime never creates a sandbox itself. It asks the operator for one by handle, the operator writes a SandboxClaim, and the claim controller builds the pod.

flowchart LR
  subgraph agent["Agent pod"]
    runtime["runtime container"]
  end
  subgraph operator["Operator"]
    api["Sandbox API<br/>POST /v1/sandboxes/acquire"]
    ctrl["SandboxClaim controller"]
  end
  claim["SandboxClaim"]
  subgraph sbx["Sandbox pod (CLAIM-sandbox)"]
    mcp["codegen-sandbox<br/>MCP over SSE, port mcp 8080"]
    cache["/sandbox-cache<br/>emptyDir"]
  end
  pvc["workspace content PVC<br/>sandboxes/CLAIM at /workspace"]
  runtime -->|"acquire sandbox/default"| api
  api -->|"creates"| claim
  ctrl -->|"watches claim, builds Deployment, Service, NetworkPolicy"| claim
  runtime -->|"MCP tool calls"| mcp
  mcp --> cache
  mcp -->|"reads and writes"| pvc
  ctrl -->|"syncs work back on reap"| pvc

Why not a tool on the agent

The obvious design is a write_file tool in the agent’s ToolRegistry. Omnia does not do this.

A tool on the AgentRuntime executes inside the runtime container: the same container that holds provider credentials, reaches the LLM, and talks to session-api and memory-api. Giving it a write primitive puts every prompt injection one step from the filesystem that container can see, and the blast radius is everything that container is trusted with.

The same objection applies to a “trusted applier” that accepts a diff from the agent and writes it. Whatever validates the diff becomes the security boundary, and a validator for arbitrary file content is hard to get right.

So the write primitive lives somewhere that is trusted with nothing else: its own pod, with its own network namespace and NetworkPolicy. See Sandbox security for why that cannot be a sidecar, and for what the pod can reach.

The shell, and what it means

The chart’s codegen-sandbox ToolRegistry allowlists the full codegen surface, Bash and the run_* tools included (sandbox.allowedTools). The class names an image and its ports; it grants no tools. The full surface is on offer because an agent that cannot build what it wrote cannot check it. A model that can execute can do anything the image can do, so a tool allowlist stops being a security boundary here. What contains a codegen sandbox is its pod: see A tool allowlist is not a boundary.

The sandbox binary’s -enable-exec, -enable-ssh and -enable-port-forward flags are never passed, and -secrets-dir is left unset. Each would open a surface that an agent with a shell would find.

A codegen pod runs the restricted profile minus readOnlyRootFilesystem, because writing is its job.

What it writes

Two volumes, and the split between them matters. Both are the chart’s codegen class’s own filesystem (sandbox.codegenClass in the Helm values): the operator builds what the class declares and knows nothing about codegen.

The workspace

The workspace is mounted at /workspace: a per-claim subtree of the workspace content PVC, at <workspace>/<namespace>/sandboxes/<claim>/. This is the sandbox’s -workspace, so it is the only thing the agent’s file tools can see.

The claim name in that path makes concurrency safe. Two datagen jobs against one agent would otherwise share a filesystem and corrupt each other, and so sandboxes are claimed per session rather than switched on per runtime.

It is a different subtree from project content (.arena/versions/...), so workspace-content isolation is untouched and the arena worker’s read-only content mount does not change. Because every part of the path is stable, the filesystem survives the pod: you can kill and resume a sandbox without losing work. A replacement claim gets a fresh directory unless the ToolRegistry sandbox sets workspace: session, which gives every claim of a session and handle one shared directory; see Session workspaces.

The toolchain cache

The cache is mounted at /sandbox-cache: a scratch emptyDir bounded at 4Gi, with the class’s env pointing HOME, GOPATH, GOCACHE and GOMODCACHE at it.

The pod runs as the restricted profile’s uid while the image sets its Go environment under its own user’s home. Inherit that and the first thing an agent does after writing a file, building it, dies with failed to initialize build cache: permission denied, which is not a failure an agent can reason its way out of.

Two things about that volume are security-relevant rather than operational:

It is outside /workspace, so it is outside -workspace and therefore outside the agent’s file tools. A cache the agent can list and edit is a cache it can sweep into whatever it produces.
It is a size-limited emptyDir, rather than another subtree of the shared PVC. A module cache runs to hundreds of megabytes and nothing collects it, so persisting it would grow the volume every workspace shares without bound; and without the limit a runaway build fills the node’s ephemeral storage and evicts its neighbours. The cost is a cold cache after a restart, which costs nothing on a sandbox with no egress to re-download from.

The Deployment runs exactly one replica with the Recreate strategy. Two pods on the same subPath would be two writers on one filesystem, and a rolling update would create that on every spec change. The price is availability: a node loss restarts the sandbox with a cold cache, and an upgrade is a brief outage.

Getting changes back out

By default the sandbox writes its own subtree and cannot reach anything that would publish it, because the NetworkPolicy denies all its egress. Moving content from there into project content is a separate mechanism acting on the PVC, outside the sandbox’s reach whatever its egress configuration: a sandbox that could publish its own output would be a sandbox that could exfiltrate.

Naming a publish-capable Service in a SandboxClass’s allowedServices reopens that route out. Omnia proves the sandbox reaches only what the class names, not what that Service does once reached (see What a sandbox can reach). Whether it accepts writes at all is your decision.

Who writes the claim

A SandboxClaim asks for one sandbox for one session. An AgentRuntime’s spec.sandbox says whether that runtime may hold any and how many, the idle timeout, and which Connections are bound to its sandboxes’ roles. The runtime cannot write claims. It asks the operator’s Sandbox API for the sandbox with handle <sandbox>/default in the conversation’s session, and the request names the handle and nothing about what the sandbox is. The operator reads the sandbox from the AgentRuntime’s own ToolRegistry, creates the claim with that sandbox’s class and target path (or finds the existing one), waits for it to bind, and returns its addresses, one per class port.

The agent’s ServiceAccount has no permissions on sandboxclaims. The Sandbox API admits a caller only when its ServiceAccount token (checked by TokenReview) is the one the named AgentRuntime’s pod runs as, in that runtime’s namespace, so an agent can ask only for its own runtime’s sandboxes. The operator takes the class from the sandbox, applies maxPerSession and the sandbox’s maxInstances, and writes the claim. A compromised runtime cannot choose a class, a sync-back target, a mount or a namespace.

The identity is the ServiceAccount, not the pod. When a Workspace gives its runtimes one shared runtime ServiceAccount (the keyless cloud-identity case), TokenReview cannot tell those runtimes apart, so each can ask for, and renew, the others’ sandboxes. That is the same trust the delegation mint already extends to runtimes that share a ServiceAccount. Give each agent its own ServiceAccount where that matters.

The fields that declare a sandbox, the class kinds and the Sandbox API are documented in SandboxClaim and the ToolRegistry’s spec.sandboxes.

Claim lifecycle

Every message of a conversation asks for the same handle, so it reaches the same sandbox, and a reconnect finds the files it left. Because the runtime cannot tell an abandoned conversation from a user who is still typing (nothing on the wire says a conversation is over), claim lifetime belongs to the reaper rather than to the message that used the sandbox.

A claim past the runtime’s maxConcurrent queues and binds when a slot frees. The agent already treats a sandbox tool call as long-running, so waiting is a state it handles. A new handle past maxPerSession, or past the sandbox’s maxInstances, is refused rather than queued, because queuing it would leave the caller waiting until another claim is reaped and ending in a timeout that says nothing. Each operator replica checks those two limits against its own cache, so the controller, which runs only on the leader, enforces them too: the claims two replicas let through at once are admitted oldest first, and the rest are refused and deleted. This depends on leader election, which the chart enables by default; with it off and more than one replica, every replica runs the controller.

Three behaviours follow from that design.

A Failed claim is replaced, once. Left in place it would refuse every later message of that conversation, long after the cause was fixed. The operator creates the replacement with the same handle and mount, including the directory it recorded for a configured mount role, so a project sandbox stays a project sandbox. It takes the sandbox’s current class and target path, so a fix to the registry reaches the replacement. A second failure is reported to the agent as a tool error, carrying the operator’s own reason.

Nothing deletes a claim when a message ends. The runtime asks the operator to renew lastUsedAt on the claim each successful tool call used. The operator writes it at most once per half the idle timeout, because a status write per tool call would be write amplification on the hottest path there is. A Bound claim nothing has touched for idleTimeout is synced back and deleted, taking its sandbox with it. A Queued claim is reaped too, after the larger of idleTimeout and 10 minutes, so an abandoned waiter does not take the next free slot; the phases and their budgets are in Reaping.

Deleting the AgentRuntime deletes its sandboxes. Claims are owned by their AgentRuntime, so garbage collection removes them, and it does not run the reaper’s forced sync-back. Work nobody promoted is lost.

Trade-offs

Putting the sandbox behind an operator-written claim costs a cold start on the first tool call of a conversation: an image pull, a pod schedule, and a wait behind the ceiling can all land inside one call. Setting allocation: eager on the ToolRegistry sandbox moves that cost to the moment the session opens, at the price of claiming sandboxes for sessions that never use a tool.