Skip to content

Give an agent a browser

Give an agent a headless browser it drives through structured tools, limited to the sites you name.

The browser runs in a sandbox pod that reaches only the hosts of the Connections bound to it. A ToolPolicy then judges each navigation by the host it asks for. Sandbox security explains what each layer contains.

  • Enterprise enabled in the chart (enterprise.enabled: true), for the ToolPolicy step. See Licensing.
  • The namespace your AgentRuntime runs in. The chart installs the ToolRegistry there, and it must already exist.
  • A PromptPack whose prompt lists the browser tools in tools. The model is offered only the tools the prompt lists.
  • A CNI that enforces NetworkPolicy. Without one, a sandbox has unrestricted egress.
  1. Set the chart values and upgrade the release:

    sandbox:
    enabled: true
    toolRegistry:
    namespace: agents
    browserClass:
    enabled: true
    runtimeClassName: gvisor # where your nodes run gVisor or Kata
    toolRegistry:
    enabled: true
    Terminal window
    helm upgrade omnia oci://ghcr.io/altairalabs/charts/omnia -n omnia-system \
    --reuse-values -f browser.yaml

    This installs the browser ClusterSandboxClass and the browser-sandbox ToolRegistry in agents. The registry’s one sandbox, browser, names the class. Its one handler offers the tools in sandbox.browserClass.toolRegistry.allowedTools, which default to every tool the browser serves. Narrow that list to offer the model fewer. See Browser Sandbox Class for every value.

    Leave runtimeClassName empty where no sandboxed runtime is installed: a RuntimeClass the cluster does not define fails the pod.

  2. Check that the registry resolved its sandbox:

    Terminal window
    kubectl get toolregistry browser-sandbox -n agents \
    -o jsonpath='{.status.conditions[?(@.type=="SandboxesValid")].status}'

    The output is True.

  1. Create a Connection for the sites, with credential.kind: none. It adds its hosts to the sandbox’s egress and carries no credential.

    apiVersion: omnia.altairalabs.ai/v1alpha1
    kind: Connection
    metadata:
    name: example-docs
    namespace: agents
    spec:
    hosts: [docs.example.com, www.example.com]
    credential:
    kind: none

    Add every host the pages load from, such as a CDN for styles and scripts. A leading *. matches any subdomain but not the apex.

  2. Reference the registry from the AgentRuntime and bind the Connection to the class’s site role:

    apiVersion: omnia.altairalabs.ai/v1alpha1
    kind: AgentRuntime
    metadata:
    name: research-agent
    namespace: agents
    spec:
    toolRegistryRef:
    name: browser-sandbox
    sandbox:
    enabled: true
    connections:
    - sandbox: browser
    role: site
    connection: example-docs
    # promptPackRef, providers and facades as for any agent

    To browse signed in, bind a browserSession Connection to site instead, then sign in to the site. Until a session is stored, every browser tool answers connection_not_enrolled.

agent_browser_open, agent_browser_read and agent_browser_tab_new take a url. A ToolPolicy can derive the host from it and permit only the hosts you list. With Enterprise, the policy broker denies every call no ToolPolicy permits, so this policy also grants the registry’s other tools.

  1. Apply a ToolPolicy that selects the whole registry:

    apiVersion: omnia.altairalabs.ai/v1alpha1
    kind: ToolPolicy
    metadata:
    name: browser-hosts
    namespace: agents
    spec:
    selector:
    registry: browser-sandbox
    producers:
    - name: hasURL
    type: cel
    cel: 'has(body.url)'
    - name: host
    type: cel
    cel: >-
    has(body.url) && body.url.split("/").size() > 2
    ? body.url.split("/")[2].split(":")[0].lowerAscii()
    : ""
    policy: |
    @id("browse-listed-hosts")
    permit (principal, action == Action::"invoke",
    resource in Registry::"browser-sandbox")
    when {
    !context.hasURL ||
    ["docs.example.com", "www.example.com"].contains(context.host)
    };
    mode: enforce
    onFailure: deny

    The host producer takes the text between // and the next /, without the port. A URL whose authority holds anything else, such as a user name or a query, yields a host that is not in the list, so the call is denied. about:blank has no host and is denied too.

  2. Keep the list equal to the Connection’s hosts, or narrower. The policy judges only the URL the model asks for. A redirect or a clicked link carries no url argument, and the Connection’s hosts bound those.

See Configure tool policies to try the policy in mode: audit first.

  1. Check the AgentRuntime’s binding:

    Terminal window
    kubectl get agentruntime research-agent -n agents \
    -o jsonpath='{.status.conditions[?(@.type=="SandboxConnectionsValid")].status}'

    The output is True.

  2. Start a conversation and ask the agent to open a page on a listed host. The first browser tool call claims the sandbox:

    Terminal window
    kubectl get sandboxclaims -n agents
    kubectl get deployments -n agents | grep egress

    The claim reaches phase Bound, and a <claim>-egress Deployment serves its egress proxy.

  3. Ask the agent to open a page on a host that is not listed. The policy broker denies the call before it reaches the sandbox.

What you seeCause and fix
SandboxConnectionsValid is False with ConnectionRoleUndeclaredThe binding names a role the class does not declare. The chart’s browser class declares site
SandboxConnectionsValid is False with ConnectionSandboxUnknownsandbox in the binding is not browser, the registry’s sandbox name
Every browser tool call is denied with no ToolPolicy permits this callNo ToolPolicy selects browser-sandbox in the agent’s namespace
A page loads without its styles, or fails to loadThe page loads from a host the Connection does not name, and the egress proxy refuses it. Add the host to hosts
The claim never reaches BoundCheck the sandbox Deployment’s events. A runtimeClassName the cluster does not define stops its pod being created. Set it empty, or install the runtime