Client-side (browser) tools

Define a client handler in a ToolRegistry so the connected browser runs the tool. For where client tools run and why only they appear on the WebSocket, see the tool execution model.

You cannot create a client handler from the dashboard. Write the YAML.

Prerequisites

An AgentRuntime with a websocket facade. Other facades cannot answer a client tool call.
A client that handles tool_call messages. The dashboard console does; for your own client, see the WebSocket protocol.

When to use a client tool

Reading something only the browser has: geolocation, camera, clipboard, local files
Driving the page: navigation, filling a form, showing a UI element
Anything that must run on the user’s device rather than server-side

Steps

Define the handler. A client handler requires a tool (name, description, inputSchema) and takes no endpoint.

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ToolRegistry
metadata:
  name: browser-tools
  namespace: agents
spec:
  handlers:
    - name: geolocation
      type: client
      tool:
        name: get_location
        description: "Read the user's current location from the browser"
        inputSchema:
          type: object
      clientConfig:
        consentMessage: "Share your location with the agent?"
        categories: [location]

clientConfig is optional. Its consentMessage and categories travel on the tool_call as consent_message and categories, replacing any the PromptPack tool declares.

See the ToolRegistry reference for every handler field.

List the tool name in the tools of each PromptPack prompt that may use it. The model is offered only the tools a prompt lists.

Reference the registry from the AgentRuntime:

spec:
  toolRegistryRef:
    name: browser-tools
  facades:
    - type: websocket

Register a handler in the client under the same tool name (get_location here). In the dashboard console, a tool with no registered handler is approved without running anything.

Verify

Open the agent in the dashboard console and send a message that makes the model call the tool. The console shows the call awaiting your approval. You can allow it once, always allow it, or deny it.

Allow once runs the handler and returns its result to the agent.
Always allow runs it, and the console runs that tool without asking from then on. The decision is stored per tool name in the browser’s localStorage.
Deny returns the rejection Denied by user to the agent.

The session’s tool call record shows how each call ended. A result is recorded as success, and an error from the client’s handler as error. A call the person denied is recorded as denied, and so is a call the client never answered. A call that is never answered does not count as consent: the console remembers no “always allow” from it, and the next call asks again.

Troubleshooting

The call is denied with client did not acknowledge tool call within timeout: the client did not send tool_call_ack or tool_result in time.
The call is denied with client tool timeout (1m0s): the client acknowledged the call but did not return a result. The model is told and the turn continues. Raise clientToolTimeout on the facade in the AgentRuntime if the tool legitimately takes longer.
The call is denied with client disconnected: the browser closed the socket before the call arrived.
The model never calls the tool: check that a prompt lists it in tools.