# WebSocket protocol

The agent facade serves a JSON-over-WebSocket protocol on `/ws`. The
machine-readable definition is
[`api/websocket/asyncapi.yaml`](https://github.com/AltairaLabs/Omnia/blob/main/api/websocket/asyncapi.yaml).
The Go types are in `internal/facade/protocol.go`.

## Connection

### URL format

```text
ws://host:port/ws?agent=<agent-name>&namespace=<namespace>&binary=<true|false>
```

| Parameter | Required | Description |
|-----------|----------|-------------|
| `agent` | Yes | Name of the AgentRuntime. Required unless the facade pod sets `OMNIA_AGENT_NAME`, which the facade uses when the parameter is absent. |
| `namespace` | No | Namespace. When absent the facade uses `OMNIA_NAMESPACE`, and `default` when that is unset. |
| `binary` | No | Enable binary WebSocket frame support (defaults to `false`) |
| `resume` | No | Session ID of a parked realtime session to re-attach to. `connected.resumed` is `true` when the session was found and `false` when the facade opened a new one. |
| `memory_facet` | No | The container (a project, client or matter) the connection's agent memory is scoped to. See [Memory tools](/reference/platform/memory-tools/#scoping-an-agent-conversation-to-a-container). 1 to 128 ASCII letters, digits and `. _ : / -`, starting with a letter or digit; a malformed or empty value, or the parameter given twice, refuses the upgrade with `400`. When the AgentRuntime sets `spec.memory.facet`, omitting it uses the agent's container and naming another refuses the upgrade with `403`. Echoed as `connected.memory_facet`. |

### Example

```bash
websocat "ws://localhost:8080/ws?agent=my-agent&namespace=production"
```

## Message types

### Client messages

Messages sent from client to server.

#### Message

Send a user message to the agent:

```json
{
  "type": "message",
  "content": "Hello, how are you?",
  "session_id": "optional-session-id",
  "metadata": {
    "user_id": "user-123"
  }
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | Must be `"message"` |
| `content` | string | No | User message content (text-only) |
| `parts` | array | No | Multi-modal content parts (see below) |
| `session_id` | string | No | Omitted, or the ID from `connected`: the message joins the connection's session. Another ID resumes that session (see [Session handling](#session-handling)) |
| `metadata` | object | No | String key-value pairs, forwarded unchanged to the runtime. The model sees them only when the agent's Provider sets `messageMetadataTemplate` (see [Message metadata](#message-metadata)) |
| `declare` | object | No | What the session needs, such as `{"sandboxes": {"<handle>": {"mounts": [{"role": "project", "id": "<project-id>"}]}}}`. Read only on the message that opens the session and ignored on later messages. At most 16 KiB; a larger value is answered with `INVALID_MESSAGE`. A refused declaration is answered with `SESSION_DECLARATION_REFUSED`. A `sandboxes` key is `<sandbox>/<instance>`, or `<sandbox>` for the default instance, and `<sandbox>` is a sandbox in the agent's ToolRegistry `spec.sandboxes`. The facade forwards the value to the runtime unread. Only the WebSocket facade carries a declaration: the A2A, REST and MCP facades send none. |

Provide `content` or `parts`. The runtime uses the text parts of `parts`, joined in order and separated by a blank line, when any part carries text; otherwise it uses `content`. Media parts are sent to the model in addition.

#### Message metadata

The runtime forwards `metadata` to the model only when the agent's default Provider sets [`messageMetadataTemplate`](/reference/core/provider/#messagemetadatatemplate). The template is rendered from the metadata and the text is appended to the message in the request. Without a template the model does not see the metadata and nothing records it.

| Rule | Value |
|------|-------|
| Reserved keys | `mock_scenario`, `content_type` and any key beginning `omnia.`. The runtime reads them itself and never forwards them |
| Keys | At most 32, each 1 to 64 characters of letters, digits, `_`, `.` or `-` |
| Values | At most 8 KiB each |
| Total | At most 32 KiB, keys and values together |

A message over a limit fails with an `INVALID_MESSAGE` error that names the limit, and does not reach the model.

The forwarded metadata is recorded with the session as a `message.metadata` runtime event. The recorded user message holds the text the client sent, without the rendered template. Session privacy treats the event's values as content, as it treats message text: PII redaction rewrites them, an opt-out suppresses them, and at-rest encryption covers them. The metadata stays on its message in the conversation context, so the model sees it with that message on every later turn. Only text messages forward metadata; a duplex audio session forwards none.

#### Multi-modal message

Send a message with images or other media:

```json
{
  "type": "message",
  "session_id": "sess-abc123",
  "parts": [
    {
      "type": "text",
      "text": "What's in this image?"
    },
    {
      "type": "image",
      "media": {
        "url": "https://example.com/photo.jpg",
        "mime_type": "image/jpeg"
      }
    }
  ]
}
```

##### ContentPart types

| Type | Description |
|------|-------------|
| `text` | Plain text content |
| `image` | Image (JPEG, PNG, GIF, WebP) |
| `audio` | Audio file (MP3, WAV, OGG) |
| `video` | Video file (MP4, WebM) |
| `file` | Generic file attachment |

##### ContentPart structure

```typescript
interface ContentPart {
  type: "text" | "image" | "audio" | "video" | "file"
  text?: string        // For type: "text"
  media?: MediaContent // For media types
}

interface MediaContent {
  // Data source (exactly one required)
  data?: string        // Base64-encoded (< 256KB recommended)
  url?: string         // HTTP/HTTPS URL
  storage_ref?: string // Backend storage reference

  // Required
  mime_type: string    // e.g., "image/jpeg", "audio/mp3"

  // Optional metadata
  filename?: string
  size_bytes?: number

  // Image-specific
  width?: number
  height?: number
  detail?: "low" | "high" | "auto"  // Vision model hint

  // Audio/Video-specific
  duration_ms?: number
  sample_rate?: number  // Audio: Hz
  channels?: number     // Audio: 1=mono, 2=stereo
}
```

##### Example: image with Base64 data

```json
{
  "type": "message",
  "parts": [
    { "type": "text", "text": "Describe this image" },
    {
      "type": "image",
      "media": {
        "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR...",
        "mime_type": "image/png"
      }
    }
  ]
}
```

#### Upload request

Request an upload URL for a file (requires facade media storage to be enabled):

```json
{
  "type": "upload_request",
  "session_id": "sess-abc123",
  "upload_request": {
    "filename": "photo.jpg",
    "mime_type": "image/jpeg",
    "size_bytes": 102400
  }
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | Must be `"upload_request"` |
| `session_id` | string | No | Resume existing session |
| `upload_request.filename` | string | Yes | Original filename |
| `upload_request.mime_type` | string | Yes | MIME type of the file |
| `upload_request.size_bytes` | number | Yes | File size in bytes |

The server responds with an `upload_ready` message containing the upload URL. See [Upload media to an agent](/how-to/agents/upload-media-to-an-agent/) for the full procedure.

#### Tool result

Answers a `tool_call` for a client-side tool:

```json
{
  "type": "tool_result",
  "tool_result": {
    "call_id": "tc-123",
    "result": {"temperature": "72F"}
  }
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | Must be `"tool_result"` |
| `tool_result.call_id` | string | Yes | The `tool_call.id` being answered |
| `tool_result.result` | any | No | Tool result (any JSON value) |
| `tool_result.error` | string | No | Set when the client's tool failed. The call is recorded as `error`. To decline a call, send `tool_call_nack` instead |

#### Tool call acknowledgement and rejection

`tool_call_ack` tells the server the client received a `tool_call` and is working on it. `tool_call_nack` declines it; the server delivers the rejection to the runtime carrying `reason`, and the session records the call as `denied`.

A call the client does not acknowledge within the ack timeout, or does not answer within `clientToolTimeout` after acknowledging it, is also recorded as `denied`. The server sends the client a `tool_result` with an `error` for that call, so the client stops waiting on it, and the turn continues.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | `"tool_call_ack"` or `"tool_call_nack"` |
| `tool_call_ack.call_id` | string | Yes | The `tool_call.id` (on `tool_call_ack`) |
| `tool_call_nack.call_id` | string | Yes | The `tool_call.id` (on `tool_call_nack`) |
| `tool_call_nack.reason` | string | Yes | Why the client cannot or will not handle the call |

#### Hangup

`{"type": "hangup"}` signals an intentional end of a realtime audio session, so the facade does not park it for re-attachment.

#### Disclosure acknowledgement

`{"type": "disclosure_ack"}` asserts that the client rendered the AI disclosure it received in `connected.ai_disclosure`. The facade records it on the session trace as `ai.disclosure.displayed` with `assertedBy: client`. The message is optional, and its absence means unacknowledged.

#### Credential refresh

`{"type": "credential_refresh", "token": "<jwt>"}` replaces the credential of a management-plane or OIDC connection. On a management-plane connection the dashboard proxy sends it, and the facade accepts it only when the token names the same end user, agent and workspace as the credential it replaces. On an OIDC connection the client sends it with a new token from its identity provider, and the facade accepts it only when the agent's OIDC validator admits the token for the same end user and, with `externalAuth.oidc.sessionClaim` set, the same session-binding claim value. The new token must also expire later than the current one. The facade ignores any other token, including a repeat of the current one. It sends no reply and does not forward the frame to the runtime.

### Server messages

Messages sent from server to client.

#### Connected

Sent once per connection, as the first frame after the socket opens, and never again on it:

```json
{
  "type": "connected",
  "session_id": "sess-abc123",
  "connected": {
    "capabilities": {
      "binary_frames": false,
      "max_payload_size": 16777216,
      "protocol_version": 1
    }
  }
}
```

The `connected` message always carries `connected.capabilities`. `binary_frames` is `true` only when the URL has `binary=true`; `max_payload_size` and `protocol_version` are always set.

| Field | Type | Description |
|-------|------|-------------|
| `connected.capabilities.binary_frames` | boolean | Server supports binary WebSocket frames |
| `connected.capabilities.max_payload_size` | number | Maximum payload size in bytes |
| `connected.capabilities.protocol_version` | number | Binary protocol version |
| `connected.resumed` | boolean | `true` when the connection re-attached to a parked realtime session |
| `connected.ai_disclosure` | boolean | The PromptPack's `requires_ai_disclosure` declaration. Absent when the pack declares nothing; an absent field is not `false`. |

#### Chunk

Streaming response chunk:

```json
{
  "type": "chunk",
  "content": "Hello! I'm doing"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `content` | string | Streamed text |
| `role` | string | `"user"` for the caller's transcribed speech on a realtime audio session; absent for assistant text |

#### Done

Final response completion:

```json
{
  "type": "done",
  "content": "Hello! I'm doing great, thank you for asking!"
}
```

#### Multi-modal response

For responses containing media (e.g., generated images), the server uses the `parts` array:

```json
{
  "type": "done",
  "session_id": "sess-abc123",
  "parts": [
    {
      "type": "text",
      "text": "Here's the image you requested:"
    },
    {
      "type": "image",
      "media": {
        "url": "https://storage.example.com/generated/img-123.png",
        "mime_type": "image/png",
        "width": 1024,
        "height": 1024
      }
    }
  ]
}
```

When `parts` is present, it takes precedence over `content`. For backward compatibility, text-only responses may use either format.

#### Tool call

The agent is calling a client-side tool. The runtime executes server-side tools and never forwards them, so `tool_call` is the only tool request in the stream.

```json
{
  "type": "tool_call",
  "tool_call": {
    "id": "tc-123",
    "name": "weather",
    "arguments": {
      "location": "San Francisco"
    }
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `tool_call.id` | string | Unique identifier of the call |
| `tool_call.name` | string | Tool name |
| `tool_call.arguments` | object | Tool arguments |
| `tool_call.consent_message` | string | Human-readable consent prompt |
| `tool_call.categories` | array of string | Semantic consent categories, such as `location` or `filesystem` |

#### Tool result

The facade sends a server `tool_result` message only when it runs the demo handler (`handler mode demo`). The runtime handler never sends it: server-side tool results stay in the runtime. For a client-side tool, the client sends `tool_result` with `tool_result.call_id` and `tool_result.result` or `tool_result.error`.

```json
{
  "type": "tool_result",
  "tool_result": {
    "id": "tc-123",
    "result": "72°F, Sunny"
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `tool_result.id` | string | The `tool_call.id` this result is for |
| `tool_result.result` | any | Tool result (any JSON value) |
| `tool_result.error` | string | Error message, set when the tool failed |

#### Upload ready

Response to an `upload_request` with the upload URL:

```json
{
  "type": "upload_ready",
  "session_id": "sess-abc123",
  "upload_ready": {
    "upload_id": "upl-xyz789",
    "upload_url": "http://agent.example.com/media/upload/upl-xyz789",
    "storage_ref": "omnia://sessions/sess-abc123/media/med-def456",
    "expires_at": "2025-01-09T12:00:00Z",
    "method": "PUT",
    "headers": {"Content-Type": "image/jpeg"}
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `upload_ready.upload_id` | string | Unique upload identifier |
| `upload_ready.upload_url` | string | URL to send the file content to |
| `upload_ready.storage_ref` | string | Storage reference for the uploaded file |
| `upload_ready.expires_at` | string | When the upload URL expires (ISO 8601) |
| `upload_ready.method` | string | HTTP method for the upload. `PUT` for every storage backend. Absent if the backend reports none; use `PUT` |
| `upload_ready.headers` | object | Headers the upload request must carry, such as `Content-Type` and, for Azure, `x-ms-blob-type: BlockBlob`. Absent if none are required |

#### Media chunk

Streaming media chunk for audio/video responses. Allows playback to begin before the entire media is generated:

```json
{
  "type": "media_chunk",
  "session_id": "sess-abc123",
  "media_chunk": {
    "media_id": "audio-xyz789",
    "sequence": 0,
    "is_last": false,
    "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVVVVVV...",
    "mime_type": "audio/mp3"
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `media_chunk.media_id` | string | Unique identifier for the media stream |
| `media_chunk.sequence` | number | Sequence number for ordering (0-indexed) |
| `media_chunk.is_last` | boolean | Whether this is the final chunk |
| `media_chunk.data` | string | Base64-encoded chunk data |
| `media_chunk.mime_type` | string | MIME type (e.g., "audio/mp3", "video/mp4") |

A client buffers chunks by `media_id` and `sequence` and assembles the complete media when `is_last: true` arrives. See [WebSocket message flows](/explanation/platform/websocket-flows/#streaming-media).

#### Interrupt

`{"type": "interrupt"}` signals a barge-in on a realtime audio session. The client discards buffered audio output and stops playback.

#### Session config

Relays the audio format the runtime negotiated for a realtime audio session. The server sends it once at the start of the session, before audio flows. The client captures audio at this format for the rest of the session.

```json
{
  "type": "session_config",
  "session_config": {
    "codec": "pcm",
    "sample_rate": 24000,
    "channels": 1
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `session_config.codec` | string | Audio encoding, such as `pcm` |
| `session_config.sample_rate` | number | Sample rate in Hz |
| `session_config.channels` | number | Channel count; `1` is mono |

#### Error

Error message:

```json
{
  "type": "error",
  "error": {
    "code": "INVALID_MESSAGE",
    "message": "Failed to parse message"
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `error.code` | string | One of the [error codes](#error-codes) |
| `error.message` | string | Error description |
| `error.details` | object | Structured detail for some codes |

## Error codes

| Code | Description |
|------|-------------|
| `INVALID_MESSAGE` | Message format is invalid, or its forwarded `metadata` breaks a limit |
| `SESSION_EXPIRED` | The session named by `session_id` cannot be resumed: its context expired, or the caller is not its owner |
| `INTERNAL_ERROR` | Internal server error |
| `UPLOAD_FAILED` | File upload operation failed |
| `MEDIA_NOT_ENABLED` | Media storage is not enabled on the facade |
| `RATE_LIMITED` | The message was shed by a rate or concurrency limit |
| `TURN_IN_PROGRESS` | The named session's previous turn is still running. The message was not processed or queued; send it again after the reply arrives. |
| `UNSATISFIABLE_FORMAT` | The runtime's duplex media counter-offer requires a format the facade does not support, such as video |
| `pack_compile_failed` | A compile-mode session did not open because its source did not compile. `error.details` holds `error` and `warnings`. The socket stays open. |
| `SESSION_DECLARATION_REFUSED` | The runtime refused the session's `declare`. `error.details` holds `type`, `handle`, `reason` and `message`. The session did not open, and the next message opens it again with whatever `declare` it carries. `reason` is the Sandbox API's own, such as `MountNotAuthorized`, `MaxInstances`, `DeclarationMismatch` or `UnknownMountRole`, or the runtime's: `UnknownSandbox`, `InvalidSandboxDeclaration`, `SandboxUnavailable`, `UnknownDeclarationType`, `InvalidDeclaration`, `SessionInitFailed` or `SessionInitUnsupported` (a runtime built without session init). |
| `MAX_ROUNDS_EXCEEDED` | The turn stopped at the agent's limit of tool-call rounds. The work so far is kept, and the next message starts a new turn. |
| `PROVIDER_RATE_LIMITED` | The model provider answered HTTP 429 and the response outlasted the runtime's retries. Retry after a short wait. |
| `PROVIDER_UNAVAILABLE` | The model provider was unreachable or answered a 5xx. 502, 503 and 504 are retried before this is sent; 500 is not, because it usually means the request itself failed. Retry shortly. |
| `PROVIDER_STALLED` | The provider's streamed response stopped partway and the idle timeout ended the turn. Retry to carry on. |
| `PROVIDER_TIMEOUT` | The model provider sent nothing for the Provider's `firstTokenTimeout` (90 seconds by default), typically an endpoint that accepted the request and never answered, or a non-streaming call outlasted `requestTimeout`. Retry shortly. |
| `PROVIDER_AUTH_FAILED` | The provider answered 401 or 403 to the runtime's credentials. Fix the Provider or its workload identity; a retry does not help. |
| `CREDENTIAL_EXPIRED` | The message arrived after the connection's management-plane or OIDC credential deadline and was not processed |
| `DUPLEX_ERROR` | A realtime audio session failed |

The `PROVIDER_*` messages name the model and its provider type, never the provider's response, which can carry credentials. Any other failed turn reaches the client as `INTERNAL_ERROR` with a generic message.

### Close codes

| Code | Reason | Meaning |
|------|--------|---------|
| `4401` | `credential expired` | A management-plane or OIDC credential expired without being refreshed. The facade closes the connection 30 seconds before the token's expiry (half the token's remaining life when that is shorter), once no turn is running. |

## Message flow

See [WebSocket message flows](/explanation/platform/websocket-flows/) for the order of messages in a new conversation, a tool call, a file upload, a media stream and a session resume.

## Session handling

### New session

The facade creates a session for every connection and announces its ID in `connected`, the first frame on the socket. A message without `session_id` joins that session:

```json
{"type": "message", "content": "Hello"}
```

The client does not have to wait for `connected` before it sends, and the facade does not send `connected` again. The frames of the reply carry the session's `session_id`.

### Resume session

Include `session_id` to resume:

```json
{
  "type": "message",
  "session_id": "sess-abc123",
  "content": "Continue our conversation"
}
```

If the session exists and hasn't expired, conversation history is preserved. A session ID the connection did not mint resumes only for the user the session archive attributes it to, as established by a credential.

When the AgentRuntime sets `externalAuth.oidc.sessionClaim`, an OIDC connection can resume only a session that the archive records as bound to the same claim value. Any other resume gets `SESSION_EXPIRED`, including when session-api is not deployed or cannot be read. A session the facade ran unrecorded, because session-api could not record it, cannot be resumed by a bound client either.

### Session expiration

Resume depends on the AgentRuntime `spec.context.ttl` (duration, default `24h`), the time-to-live of the context store entries.

Resuming a session whose context has expired returns the `SESSION_EXPIRED` error with the message `session context has expired; start a new session`. The message is not processed. The client sends it again without `session_id`, and it joins the connection's own session; the reply's frames carry that session's ID. A `session_id` the archive has never seen opens as a new session.

## Media upload (optional)

When facade media storage is enabled, clients can upload files via HTTP before referencing them in WebSocket messages. This avoids base64-encoding large files in the WebSocket protocol. See [Upload media to an agent](/how-to/agents/upload-media-to-an-agent/) for the procedure.

Media storage is configured on the AgentRuntime at `spec.media.storage`; `none` is the default and disables it.

The routes are served on the facade's HTTP port, outside `/ws`.

| Route | Storage | Description |
|-------|---------|-------------|
| `POST /media/request-upload` | All | Request an upload URL |
| `PUT /media/upload/{upload-id}` | `local` | Upload file content |
| `POST /media/confirm-upload/{upload-id}` | `s3`, `gcs`, `azure` | Confirm a direct upload |
| `GET /media/info/{session-id}/{media-id}` | All | Read media metadata |
| `GET /media/download/{session-id}/{media-id}` | All | Download media |

### Request upload

`POST /media/request-upload` takes a JSON body:

| Field | Type | Description |
|-------|------|-------------|
| `session_id` | string | Session the media belongs to |
| `filename` | string | Original filename |
| `mime_type` | string | MIME type of the file |
| `size_bytes` | number | File size in bytes |

The response is a JSON object. Its keys match the WebSocket `upload_ready` message:

| Field | Type | Description |
|-------|------|-------------|
| `upload_id` | string | Unique upload identifier |
| `upload_url` | string | URL to send the file content to |
| `storage_ref` | string | Storage reference to name in a message part |
| `expires_at` | string | When the upload URL expires (ISO 8601) |
| `headers` | object | Headers the upload request must carry, such as `Content-Type`, and `x-ms-blob-type` for Azure |
| `method` | string | HTTP method for the upload, `PUT` |

### Upload file

`PUT /media/upload/{upload-id}` is registered only for local media storage. For `s3`, `gcs` and `azure` the upload URL is a presigned cloud URL, and the client then calls `POST /media/confirm-upload/{upload-id}`.

Response: `204 No Content` on success. `POST /media/confirm-upload/{upload-id}` answers with the media metadata, in the same shape as the [media info endpoint](#media-info-endpoint).

The HTTP response is the confirmation that the upload completed. The WebSocket sends no message for it.

### Media info endpoint

Retrieve metadata about uploaded media:

```bash
GET /media/info/{session-id}/{media-id}
```

The response is a JSON object:

| Field | Type | Description |
|-------|------|-------------|
| `storage_ref` | string | Storage reference |
| `filename` | string | Original filename |
| `mime_type` | string | MIME type |
| `size_bytes` | number | File size in bytes |
| `created_at` | string | When the media was uploaded (ISO 8601) |
| `expires_at` | string | When the media is deleted (ISO 8601). Absent when it does not expire |

### Media download endpoint

Download uploaded media:

```bash
GET /media/download/{session-id}/{media-id}
```

With local storage the response is the file with `Content-Type` and `Content-Disposition` headers. With cloud storage the response is a `307` redirect to a presigned URL.

See [AgentRuntime media.storage configuration](/reference/core/agentruntime/#mediastorage) for the backend fields.

## Binary WebSocket frames

When binary frame support is enabled (`binary=true` query parameter), the server sends duplex audio as binary WebSocket frames. Base64 adds about 33% to the size of media in a JSON `media_chunk`. A binary frame carries the raw bytes plus a 32-byte header and JSON metadata. See [Handle binary WebSocket frames](/how-to/agents/handle-binary-websocket-frames/) for a client example.

### Binary frame structure

```mermaid
flowchart LR
    H["Header<br/>32 bytes"] --- M["Metadata<br/>JSON"] --- P["Binary payload"]
```

### Header layout

| Field | Offset | Size | Type | Description |
|-------|--------|------|------|-------------|
| Magic | 0 | 4 | bytes | `"OMNI"` magic bytes |
| Version | 4 | 1 | uint8 | Protocol version (`1`) |
| Flags | 5 | 1 | uint8 | Bit flags (see below) |
| MessageType | 6 | 2 | uint16 | Message type (big-endian) |
| MetadataLen | 8 | 4 | uint32 | JSON metadata length (big-endian) |
| PayloadLen | 12 | 4 | uint32 | Binary payload length (big-endian) |
| Sequence | 16 | 4 | uint32 | Sequence number (big-endian) |
| MediaID | 20 | 12 | bytes | Media stream identifier |

### Flags

| Bit | Mask | Name | Description |
|-----|------|------|-------------|
| 0 | `0x01` | Compressed | Reserved. The facade neither sets nor reads it; do not set it |
| 1 | `0x02` | Chunked | Part of a chunked transfer |
| 2 | `0x04` | IsLast | Last chunk in a stream |

A payload larger than 1 MB is split into 64 KB frames with the Chunked flag set. In a chunked transfer `Sequence` is the zero-based chunk index and the metadata carries `total_chunks`.

On a frame the client sends, IsLast marks the final frame of the entire audio call and ends the audio session. A client sets it only on the true final frame.

### Message types

| Value | Name | Description |
|-------|------|-------------|
| 1 | MediaChunk | Streaming media chunk |

Value `1` (MediaChunk) is the only binary message type. A frame with another type is answered with `INVALID_MESSAGE` and the message `unknown binary message type`.

### Binary media chunk

The metadata is a JSON object:

```json
{
  "session_id": "sess-abc123",
  "mime_type": "audio/mp3"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `session_id` | string | Session identifier |
| `mime_type` | string | MIME type of the payload |
| `total_chunks` | number | Number of chunks in a chunked transfer |
| `sample_rate` | number | Audio sample rate in Hz; set on the first frame a client sends |
| `channels` | number | Audio channel count; set on the first frame a client sends |
| `codec` | string | Audio encoding; set on the first frame a client sends |

The payload contains raw binary audio/video data (not base64-encoded).

### Fallback behavior

When a client doesn't request binary frames (`binary=true` not set), the server always sends JSON text frames with base64-encoded media data. This ensures backward compatibility with existing clients.

## Connection health

The server sends a WebSocket ping frame every ping interval. Most WebSocket client libraries answer pings with pong frames automatically.

The server closes the connection when nothing arrives from the client for the pong timeout. A pong and any other frame from the client both reset the timer.

| Setting | Default |
|---|---|
| Ping interval | 30 seconds |
| Pong timeout | 60 seconds |

## Type definitions

### JSON field names

The WebSocket protocol uses **snake_case** for JSON field names to match Go conventions:

```json
{
  "session_id": "...",
  "mime_type": "image/png",
  "size_bytes": 1024
}
```

The generated TypeScript types use snake_case property names that match the JSON (`mime_type`, `session_id`, `size_bytes`).