WebSocket message flows

A client holds one WebSocket to the agent facade and exchanges JSON messages with it. This page shows the order in which those messages arrive for the common cases. The WebSocket protocol reference lists every message and field.

A new conversation

The facade mints a session ID when the socket opens and announces it in connected, the first frame on the socket and the only connected it sends. The client does not have to wait for connected before it sends: a message without session_id joins the session the facade announced.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant R as Runtime

    C->>F: WebSocket connect
    F-->>C: connected (session_id)
    C->>F: message
    F->>R: gRPC ClientMessage
    R-->>F: chunk
    F-->>C: chunk
    R-->>F: done
    F-->>C: done

The facade runs each message as a turn in its own goroutine, so it keeps reading from the socket while the turn runs. A turn runs to its end after the client disconnects, and the facade records the reply in the session archive, where a client that resumes the session can read it. Only one turn runs per session at a time, and a second message for a busy session is answered with TURN_IN_PROGRESS.

Tool calls

Tools run in one of two places, and only one of them is visible on the socket. The runtime executes server-side tools (HTTP, MCP, memory and the like) and never forwards them. A client-side tool is the only kind that produces tool_call.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant R as Runtime

    C->>F: message
    F->>R: gRPC ClientMessage
    R-->>F: tool_call (client-side)
    F-->>C: tool_call
    C->>F: tool_call_ack
    C->>F: tool_result (call_id)
    F->>R: tool result
    R-->>F: done
    F-->>C: done

The read loop stays free during the turn so that tool_call_ack, tool_call_nack and tool_result reach the waiting turn. A tool_call_nack is delivered to the runtime as a rejection carrying the nack reason, and the call is recorded as denied. A tool_result with an error is recorded as an error. A call the client never answers is denied: the facade tells the runtime, sends the client a tool_result that ends the call, and the turn continues. See Client tools for defining one.

File upload

A large file travels out of band, through an upload URL. The socket carries only the request for that URL and the reference to use afterwards, which keeps large payloads out of the conversation stream and out of the facade’s memory. A small file can also be sent inline as base64 in a message part’s data field, with under about 256 KB recommended.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant S as Media storage

    C->>F: upload_request
    F-->>C: upload_ready (upload_url, storage_ref)
    C->>S: PUT file to upload_url
    C->>F: message with storage_ref
    F-->>C: chunk
    F-->>C: done

With local storage the upload URL is http://localhost:<facade port>/media/upload/<id>, served by the facade itself. With S3, GCS or Azure it is a presigned cloud URL, and the client confirms the upload with POST /media/confirm-upload/{upload-id}. The HTTP response that ends the upload is the client’s confirmation; the socket sends nothing for it, since that request can reach a different facade replica from the one holding the socket. See Upload media to an agent for the procedure and Configure media storage for the backends.

Streaming media

A runtime that produces audio or video sends it as a sequence of chunks, so the client can start playback before generation ends. Each chunk carries a media_id and a sequence, and the last carries is_last.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant R as Runtime

    R-->>F: media chunk 0
    F-->>C: media_chunk (sequence 0)
    Note over C: Playback can begin
    R-->>F: media chunk 1 (final)
    F-->>C: media_chunk (sequence 1, is_last)
    R-->>F: done
    F-->>C: done

A client that connects with binary=true receives duplex (voice) audio as binary frames. Media produced on the text path, and every client without binary=true, receives base64 in JSON. The binary framing trades the roughly 33% base64 overhead for a client that must parse a 32-byte header. See Handle binary WebSocket frames.

Resuming a session

A message that names a session_id the connection did not mint resumes that session. The facade checks ownership, then asks the runtime whether the conversation’s context still exists.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant R as Runtime

    C->>F: WebSocket connect
    F-->>C: connected (session_id)
    C->>F: message (session_id of an earlier session)
    F->>F: check the caller owns the session
    F->>R: does the conversation context exist?
    R-->>F: yes
    F->>R: gRPC ClientMessage
    R-->>F: done
    F-->>C: done

Session management covers the ownership check, SESSION_EXPIRED, the context store and ?resume re-attachment for realtime audio.