WebSocket protocol

The agent facade serves a JSON-over-WebSocket protocol on /ws. The machine-readable definition is api/websocket/asyncapi.yaml. The Go types are in internal/facade/protocol.go.

Connection

URL format

ws://host:port/ws?agent=<agent-name>&namespace=<namespace>&binary=<true|false>

Parameter	Required	Description
agent	Yes	Name of the AgentRuntime. Required unless the facade pod sets OMNIA_AGENT_NAME, which the facade uses when the parameter is absent.
namespace	No	Namespace. When absent the facade uses OMNIA_NAMESPACE, and default when that is unset.
binary	No	Enable binary WebSocket frame support (defaults to false)
resume	No	Session ID of a parked realtime session to re-attach to. connected.resumed is true when the session was found and false when the facade opened a new one.
memory_facet	No	The container (a project, client or matter) the connection’s agent memory is scoped to. See Memory tools. 1 to 128 ASCII letters, digits and . _ : / -, starting with a letter or digit; a malformed or empty value, or the parameter given twice, refuses the upgrade with 400. When the AgentRuntime sets spec.memory.facet, omitting it uses the agent’s container and naming another refuses the upgrade with 403. Echoed as connected.memory_facet.

Example

websocat "ws://localhost:8080/ws?agent=my-agent&namespace=production"

Message types

Client messages

Messages sent from client to server.

Message

Send a user message to the agent:

{
  "type": "message",
  "content": "Hello, how are you?",
  "session_id": "optional-session-id",
  "metadata": {
    "user_id": "user-123"
  }
}

Field	Type	Required	Description
type	string	Yes	Must be "message"
content	string	No	User message content (text-only)
parts	array	No	Multi-modal content parts (see below)
session_id	string	No	Omitted, or the ID from connected: the message joins the connection’s session. Another ID resumes that session (see Session handling)
metadata	object	No	String key-value pairs, forwarded unchanged to the runtime. The model sees them only when the agent’s Provider sets messageMetadataTemplate (see Message metadata)
declare	object	No	What the session needs, such as {"sandboxes": {"<handle>": {"mounts": [{"role": "project", "id": "<project-id>"}]}}}. Read only on the message that opens the session and ignored on later messages. At most 16 KiB; a larger value is answered with INVALID_MESSAGE. A refused declaration is answered with SESSION_DECLARATION_REFUSED. A sandboxes key is <sandbox>/<instance>, or <sandbox> for the default instance, and <sandbox> is a sandbox in the agent’s ToolRegistry spec.sandboxes. The facade forwards the value to the runtime unread. Only the WebSocket facade carries a declaration: the A2A, REST and MCP facades send none.

Provide content or parts. The runtime uses the text parts of parts, joined in order and separated by a blank line, when any part carries text; otherwise it uses content. Media parts are sent to the model in addition.

Message metadata

The runtime forwards metadata to the model only when the agent’s default Provider sets messageMetadataTemplate. The template is rendered from the metadata and the text is appended to the message in the request. Without a template the model does not see the metadata and nothing records it.

Rule	Value
Reserved keys	mock_scenario, content_type and any key beginning omnia.. The runtime reads them itself and never forwards them
Keys	At most 32, each 1 to 64 characters of letters, digits, _, . or -
Values	At most 8 KiB each
Total	At most 32 KiB, keys and values together

A message over a limit fails with an INVALID_MESSAGE error that names the limit, and does not reach the model.

The forwarded metadata is recorded with the session as a message.metadata runtime event. The recorded user message holds the text the client sent, without the rendered template. Session privacy treats the event’s values as content, as it treats message text: PII redaction rewrites them, an opt-out suppresses them, and at-rest encryption covers them. The metadata stays on its message in the conversation context, so the model sees it with that message on every later turn. Only text messages forward metadata; a duplex audio session forwards none.

Multi-modal message

Send a message with images or other media:

{
  "type": "message",
  "session_id": "sess-abc123",
  "parts": [
    {
      "type": "text",
      "text": "What's in this image?"
    },
    {
      "type": "image",
      "media": {
        "url": "https://example.com/photo.jpg",
        "mime_type": "image/jpeg"
      }
    }
  ]
}

ContentPart types

Type	Description
text	Plain text content
image	Image (JPEG, PNG, GIF, WebP)
audio	Audio file (MP3, WAV, OGG)
video	Video file (MP4, WebM)
file	Generic file attachment

ContentPart structure

interface ContentPart {
  type: "text" | "image" | "audio" | "video" | "file"
  text?: string        // For type: "text"
  media?: MediaContent // For media types
}

interface MediaContent {
  // Data source (exactly one required)
  data?: string        // Base64-encoded (< 256KB recommended)
  url?: string         // HTTP/HTTPS URL
  storage_ref?: string // Backend storage reference

  // Required
  mime_type: string    // e.g., "image/jpeg", "audio/mp3"

  // Optional metadata
  filename?: string
  size_bytes?: number

  // Image-specific
  width?: number
  height?: number
  detail?: "low" | "high" | "auto"  // Vision model hint

  // Audio/Video-specific
  duration_ms?: number
  sample_rate?: number  // Audio: Hz
  channels?: number     // Audio: 1=mono, 2=stereo
}

Example: image with Base64 data

{
  "type": "message",
  "parts": [
    { "type": "text", "text": "Describe this image" },
    {
      "type": "image",
      "media": {
        "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR...",
        "mime_type": "image/png"
      }
    }
  ]
}

Upload request

Request an upload URL for a file (requires facade media storage to be enabled):

{
  "type": "upload_request",
  "session_id": "sess-abc123",
  "upload_request": {
    "filename": "photo.jpg",
    "mime_type": "image/jpeg",
    "size_bytes": 102400
  }
}

Field	Type	Required	Description
type	string	Yes	Must be "upload_request"
session_id	string	No	Resume existing session
upload_request.filename	string	Yes	Original filename
upload_request.mime_type	string	Yes	MIME type of the file
upload_request.size_bytes	number	Yes	File size in bytes

The server responds with an upload_ready message containing the upload URL. See Upload media to an agent for the full procedure.

Tool result

Answers a tool_call for a client-side tool:

{
  "type": "tool_result",
  "tool_result": {
    "call_id": "tc-123",
    "result": {"temperature": "72F"}
  }
}

Field	Type	Required	Description
type	string	Yes	Must be "tool_result"
tool_result.call_id	string	Yes	The tool_call.id being answered
tool_result.result	any	No	Tool result (any JSON value)
tool_result.error	string	No	Set when the client’s tool failed. The call is recorded as error. To decline a call, send tool_call_nack instead

Tool call acknowledgement and rejection

tool_call_ack tells the server the client received a tool_call and is working on it. tool_call_nack declines it; the server delivers the rejection to the runtime carrying reason, and the session records the call as denied.

A call the client does not acknowledge within the ack timeout, or does not answer within clientToolTimeout after acknowledging it, is also recorded as denied. The server sends the client a tool_result with an error for that call, so the client stops waiting on it, and the turn continues.

Field	Type	Required	Description
type	string	Yes	"tool_call_ack" or "tool_call_nack"
tool_call_ack.call_id	string	Yes	The tool_call.id (on tool_call_ack)
tool_call_nack.call_id	string	Yes	The tool_call.id (on tool_call_nack)
tool_call_nack.reason	string	Yes	Why the client cannot or will not handle the call

Hangup

{"type": "hangup"} signals an intentional end of a realtime audio session, so the facade does not park it for re-attachment.

Disclosure acknowledgement

{"type": "disclosure_ack"} asserts that the client rendered the AI disclosure it received in connected.ai_disclosure. The facade records it on the session trace as ai.disclosure.displayed with assertedBy: client. The message is optional, and its absence means unacknowledged.

Credential refresh

{"type": "credential_refresh", "token": "<jwt>"} replaces the credential of a management-plane or OIDC connection. On a management-plane connection the dashboard proxy sends it, and the facade accepts it only when the token names the same end user, agent and workspace as the credential it replaces. On an OIDC connection the client sends it with a new token from its identity provider, and the facade accepts it only when the agent’s OIDC validator admits the token for the same end user and, with externalAuth.oidc.sessionClaim set, the same session-binding claim value. The new token must also expire later than the current one. The facade ignores any other token, including a repeat of the current one. It sends no reply and does not forward the frame to the runtime.

Server messages

Messages sent from server to client.

Connected

Sent once per connection, as the first frame after the socket opens, and never again on it:

{
  "type": "connected",
  "session_id": "sess-abc123",
  "connected": {
    "capabilities": {
      "binary_frames": false,
      "max_payload_size": 16777216,
      "protocol_version": 1
    }
  }
}

The connected message always carries connected.capabilities. binary_frames is true only when the URL has binary=true; max_payload_size and protocol_version are always set.

Field	Type	Description
connected.capabilities.binary_frames	boolean	Server supports binary WebSocket frames
connected.capabilities.max_payload_size	number	Maximum payload size in bytes
connected.capabilities.protocol_version	number	Binary protocol version
connected.resumed	boolean	true when the connection re-attached to a parked realtime session
connected.ai_disclosure	boolean	The PromptPack’s requires_ai_disclosure declaration. Absent when the pack declares nothing; an absent field is not false.

Chunk

Streaming response chunk:

{
  "type": "chunk",
  "content": "Hello! I'm doing"
}

Field	Type	Description
content	string	Streamed text
role	string	"user" for the caller’s transcribed speech on a realtime audio session; absent for assistant text

Done

Final response completion:

{
  "type": "done",
  "content": "Hello! I'm doing great, thank you for asking!"
}

Multi-modal response

For responses containing media (e.g., generated images), the server uses the parts array:

{
  "type": "done",
  "session_id": "sess-abc123",
  "parts": [
    {
      "type": "text",
      "text": "Here's the image you requested:"
    },
    {
      "type": "image",
      "media": {
        "url": "https://storage.example.com/generated/img-123.png",
        "mime_type": "image/png",
        "width": 1024,
        "height": 1024
      }
    }
  ]
}

When parts is present, it takes precedence over content. For backward compatibility, text-only responses may use either format.

Tool call

The agent is calling a client-side tool. The runtime executes server-side tools and never forwards them, so tool_call is the only tool request in the stream.

{
  "type": "tool_call",
  "tool_call": {
    "id": "tc-123",
    "name": "weather",
    "arguments": {
      "location": "San Francisco"
    }
  }
}

Field	Type	Description
tool_call.id	string	Unique identifier of the call
tool_call.name	string	Tool name
tool_call.arguments	object	Tool arguments
tool_call.consent_message	string	Human-readable consent prompt
tool_call.categories	array of string	Semantic consent categories, such as location or filesystem

Tool result

The facade sends a server tool_result message only when it runs the demo handler (handler mode demo). The runtime handler never sends it: server-side tool results stay in the runtime. For a client-side tool, the client sends tool_result with tool_result.call_id and tool_result.result or tool_result.error.

{
  "type": "tool_result",
  "tool_result": {
    "id": "tc-123",
    "result": "72°F, Sunny"
  }
}

Field	Type	Description
tool_result.id	string	The tool_call.id this result is for
tool_result.result	any	Tool result (any JSON value)
tool_result.error	string	Error message, set when the tool failed

Upload ready

Response to an upload_request with the upload URL:

{
  "type": "upload_ready",
  "session_id": "sess-abc123",
  "upload_ready": {
    "upload_id": "upl-xyz789",
    "upload_url": "http://agent.example.com/media/upload/upl-xyz789",
    "storage_ref": "omnia://sessions/sess-abc123/media/med-def456",
    "expires_at": "2025-01-09T12:00:00Z",
    "method": "PUT",
    "headers": {"Content-Type": "image/jpeg"}
  }
}

Field	Type	Description
upload_ready.upload_id	string	Unique upload identifier
upload_ready.upload_url	string	URL to send the file content to
upload_ready.storage_ref	string	Storage reference for the uploaded file
upload_ready.expires_at	string	When the upload URL expires (ISO 8601)
upload_ready.method	string	HTTP method for the upload. PUT for every storage backend. Absent if the backend reports none; use PUT
upload_ready.headers	object	Headers the upload request must carry, such as Content-Type and, for Azure, x-ms-blob-type: BlockBlob. Absent if none are required

Media chunk

Streaming media chunk for audio/video responses. Allows playback to begin before the entire media is generated:

{
  "type": "media_chunk",
  "session_id": "sess-abc123",
  "media_chunk": {
    "media_id": "audio-xyz789",
    "sequence": 0,
    "is_last": false,
    "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVVVVVV...",
    "mime_type": "audio/mp3"
  }
}

Field	Type	Description
media_chunk.media_id	string	Unique identifier for the media stream
media_chunk.sequence	number	Sequence number for ordering (0-indexed)
media_chunk.is_last	boolean	Whether this is the final chunk
media_chunk.data	string	Base64-encoded chunk data
media_chunk.mime_type	string	MIME type (e.g., “audio/mp3”, “video/mp4”)

A client buffers chunks by media_id and sequence and assembles the complete media when is_last: true arrives. See WebSocket message flows.

Interrupt

{"type": "interrupt"} signals a barge-in on a realtime audio session. The client discards buffered audio output and stops playback.

Session config

Relays the audio format the runtime negotiated for a realtime audio session. The server sends it once at the start of the session, before audio flows. The client captures audio at this format for the rest of the session.

{
  "type": "session_config",
  "session_config": {
    "codec": "pcm",
    "sample_rate": 24000,
    "channels": 1
  }
}

Field	Type	Description
session_config.codec	string	Audio encoding, such as pcm
session_config.sample_rate	number	Sample rate in Hz
session_config.channels	number	Channel count; 1 is mono

Error

Error message:

{
  "type": "error",
  "error": {
    "code": "INVALID_MESSAGE",
    "message": "Failed to parse message"
  }
}

Field	Type	Description
error.code	string	One of the error codes
error.message	string	Error description
error.details	object	Structured detail for some codes

Error codes

Code	Description
INVALID_MESSAGE	Message format is invalid, or its forwarded metadata breaks a limit
SESSION_EXPIRED	The session named by session_id cannot be resumed: its context expired, or the caller is not its owner
INTERNAL_ERROR	Internal server error
UPLOAD_FAILED	File upload operation failed
MEDIA_NOT_ENABLED	Media storage is not enabled on the facade
RATE_LIMITED	The message was shed by a rate or concurrency limit
TURN_IN_PROGRESS	The named session’s previous turn is still running. The message was not processed or queued; send it again after the reply arrives.
UNSATISFIABLE_FORMAT	The runtime’s duplex media counter-offer requires a format the facade does not support, such as video
pack_compile_failed	A compile-mode session did not open because its source did not compile. error.details holds error and warnings. The socket stays open.
SESSION_DECLARATION_REFUSED	The runtime refused the session’s declare. error.details holds type, handle, reason and message. The session did not open, and the next message opens it again with whatever declare it carries. reason is the Sandbox API’s own, such as MountNotAuthorized, MaxInstances, DeclarationMismatch or UnknownMountRole, or the runtime’s: UnknownSandbox, InvalidSandboxDeclaration, SandboxUnavailable, UnknownDeclarationType, InvalidDeclaration, SessionInitFailed or SessionInitUnsupported (a runtime built without session init).
MAX_ROUNDS_EXCEEDED	The turn stopped at the agent’s limit of tool-call rounds. The work so far is kept, and the next message starts a new turn.
PROVIDER_RATE_LIMITED	The model provider answered HTTP 429 and the response outlasted the runtime’s retries. Retry after a short wait.
PROVIDER_UNAVAILABLE	The model provider was unreachable or answered a 5xx. 502, 503 and 504 are retried before this is sent; 500 is not, because it usually means the request itself failed. Retry shortly.
PROVIDER_STALLED	The provider’s streamed response stopped partway and the idle timeout ended the turn. Retry to carry on.
PROVIDER_TIMEOUT	The model provider sent nothing for the Provider’s firstTokenTimeout (90 seconds by default), typically an endpoint that accepted the request and never answered, or a non-streaming call outlasted requestTimeout. Retry shortly.
PROVIDER_AUTH_FAILED	The provider answered 401 or 403 to the runtime’s credentials. Fix the Provider or its workload identity; a retry does not help.
CREDENTIAL_EXPIRED	The message arrived after the connection’s management-plane or OIDC credential deadline and was not processed
DUPLEX_ERROR	A realtime audio session failed

The PROVIDER_* messages name the model and its provider type, never the provider’s response, which can carry credentials. Any other failed turn reaches the client as INTERNAL_ERROR with a generic message.

Close codes

Code	Reason	Meaning
4401	credential expired	A management-plane or OIDC credential expired without being refreshed. The facade closes the connection 30 seconds before the token’s expiry (half the token’s remaining life when that is shorter), once no turn is running.

Message flow

See WebSocket message flows for the order of messages in a new conversation, a tool call, a file upload, a media stream and a session resume.

Session handling

New session

The facade creates a session for every connection and announces its ID in connected, the first frame on the socket. A message without session_id joins that session:

{"type": "message", "content": "Hello"}

The client does not have to wait for connected before it sends, and the facade does not send connected again. The frames of the reply carry the session’s session_id.

Resume session

Include session_id to resume:

{
  "type": "message",
  "session_id": "sess-abc123",
  "content": "Continue our conversation"
}

If the session exists and hasn’t expired, conversation history is preserved. A session ID the connection did not mint resumes only for the user the session archive attributes it to, as established by a credential.

When the AgentRuntime sets externalAuth.oidc.sessionClaim, an OIDC connection can resume only a session that the archive records as bound to the same claim value. Any other resume gets SESSION_EXPIRED, including when session-api is not deployed or cannot be read. A session the facade ran unrecorded, because session-api could not record it, cannot be resumed by a bound client either.

Session expiration

Resume depends on the AgentRuntime spec.context.ttl (duration, default 24h), the time-to-live of the context store entries.

Resuming a session whose context has expired returns the SESSION_EXPIRED error with the message session context has expired; start a new session. The message is not processed. The client sends it again without session_id, and it joins the connection’s own session; the reply’s frames carry that session’s ID. A session_id the archive has never seen opens as a new session.

Media upload (optional)

When facade media storage is enabled, clients can upload files via HTTP before referencing them in WebSocket messages. This avoids base64-encoding large files in the WebSocket protocol. See Upload media to an agent for the procedure.

Media storage is configured on the AgentRuntime at spec.media.storage; none is the default and disables it.

The routes are served on the facade’s HTTP port, outside /ws.

Route	Storage	Description
POST /media/request-upload	All	Request an upload URL
PUT /media/upload/{upload-id}	local	Upload file content
POST /media/confirm-upload/{upload-id}	s3, gcs, azure	Confirm a direct upload
GET /media/info/{session-id}/{media-id}	All	Read media metadata
GET /media/download/{session-id}/{media-id}	All	Download media

Request upload

POST /media/request-upload takes a JSON body:

Field	Type	Description
session_id	string	Session the media belongs to
filename	string	Original filename
mime_type	string	MIME type of the file
size_bytes	number	File size in bytes

The response is a JSON object. Its keys match the WebSocket upload_ready message:

Field	Type	Description
upload_id	string	Unique upload identifier
upload_url	string	URL to send the file content to
storage_ref	string	Storage reference to name in a message part
expires_at	string	When the upload URL expires (ISO 8601)
headers	object	Headers the upload request must carry, such as Content-Type, and x-ms-blob-type for Azure
method	string	HTTP method for the upload, PUT

Upload file

PUT /media/upload/{upload-id} is registered only for local media storage. For s3, gcs and azure the upload URL is a presigned cloud URL, and the client then calls POST /media/confirm-upload/{upload-id}.

Response: 204 No Content on success. POST /media/confirm-upload/{upload-id} answers with the media metadata, in the same shape as the media info endpoint.

The HTTP response is the confirmation that the upload completed. The WebSocket sends no message for it.

Media info endpoint

Retrieve metadata about uploaded media:

GET /media/info/{session-id}/{media-id}

The response is a JSON object:

Field	Type	Description
storage_ref	string	Storage reference
filename	string	Original filename
mime_type	string	MIME type
size_bytes	number	File size in bytes
created_at	string	When the media was uploaded (ISO 8601)
expires_at	string	When the media is deleted (ISO 8601). Absent when it does not expire

Media download endpoint

Download uploaded media:

GET /media/download/{session-id}/{media-id}

With local storage the response is the file with Content-Type and Content-Disposition headers. With cloud storage the response is a 307 redirect to a presigned URL.

See AgentRuntime media.storage configuration for the backend fields.

Binary WebSocket frames

When binary frame support is enabled (binary=true query parameter), the server sends duplex audio as binary WebSocket frames. Base64 adds about 33% to the size of media in a JSON media_chunk. A binary frame carries the raw bytes plus a 32-byte header and JSON metadata. See Handle binary WebSocket frames for a client example.

Binary frame structure

flowchart LR
    H["Header<br/>32 bytes"] --- M["Metadata<br/>JSON"] --- P["Binary payload"]

Header layout

Field	Offset	Size	Type	Description
Magic	0	4	bytes	"OMNI" magic bytes
Version	4	1	uint8	Protocol version (1)
Flags	5	1	uint8	Bit flags (see below)
MessageType	6	2	uint16	Message type (big-endian)
MetadataLen	8	4	uint32	JSON metadata length (big-endian)
PayloadLen	12	4	uint32	Binary payload length (big-endian)
Sequence	16	4	uint32	Sequence number (big-endian)
MediaID	20	12	bytes	Media stream identifier

Flags

Bit	Mask	Name	Description
0	0x01	Compressed	Reserved. The facade neither sets nor reads it; do not set it
1	0x02	Chunked	Part of a chunked transfer
2	0x04	IsLast	Last chunk in a stream

A payload larger than 1 MB is split into 64 KB frames with the Chunked flag set. In a chunked transfer Sequence is the zero-based chunk index and the metadata carries total_chunks.

On a frame the client sends, IsLast marks the final frame of the entire audio call and ends the audio session. A client sets it only on the true final frame.

Message types

Value	Name	Description
1	MediaChunk	Streaming media chunk

Value 1 (MediaChunk) is the only binary message type. A frame with another type is answered with INVALID_MESSAGE and the message unknown binary message type.

Binary media chunk

The metadata is a JSON object:

{
  "session_id": "sess-abc123",
  "mime_type": "audio/mp3"
}

Field	Type	Description
session_id	string	Session identifier
mime_type	string	MIME type of the payload
total_chunks	number	Number of chunks in a chunked transfer
sample_rate	number	Audio sample rate in Hz; set on the first frame a client sends
channels	number	Audio channel count; set on the first frame a client sends
codec	string	Audio encoding; set on the first frame a client sends

The payload contains raw binary audio/video data (not base64-encoded).

Fallback behavior

When a client doesn’t request binary frames (binary=true not set), the server always sends JSON text frames with base64-encoded media data. This ensures backward compatibility with existing clients.

Connection health

The server sends a WebSocket ping frame every ping interval. Most WebSocket client libraries answer pings with pong frames automatically.

The server closes the connection when nothing arrives from the client for the pong timeout. A pong and any other frame from the client both reset the timer.

Setting	Default
Ping interval	30 seconds
Pong timeout	60 seconds

Type definitions

JSON field names

The WebSocket protocol uses snake_case for JSON field names to match Go conventions:

{
  "session_id": "...",
  "mime_type": "image/png",
  "size_bytes": 1024
}

The generated TypeScript types use snake_case property names that match the JSON (mime_type, session_id, size_bytes).