Session archive format

The compaction job writes archived sessions to an object store (S3, GCS or Azure Blob) as Parquet files in a Hive-partitioned layout. Session retention explains when a session is archived. Query the session archive shows how to read the files with SQL engines.

Object layout

Every object key starts with the prefix sessions/. The prefix is fixed; the compaction job has no flag or chart value for it.

sessions/
  _manifest.json
  year=2026/month=04/day=10/agent=my-agent/part-0000.parquet
  year=2026/month=04/day=10/agent=my-agent/part-0001.parquet
  year=2026/month=04/day=11/agent=another-agent/part-0000.parquet

Element	Behaviour
`year=`, `month=`, `day=`	The session’s `created_at` date in UTC. `year` has four digits, `month` and `day` have two.
`agent=`	The agent name with every character outside `[a-zA-Z0-9_-]` replaced by `_`.
`part-NNNN.parquet`	Each write to a partition starts numbering at the number of objects already in it. A write splits into several files when its estimated size passes 128 MB.
Compression	Set by the chart value `sessionRetention.compaction.compression`: `snappy` (default), `gzip`, `zstd` or `none`. Parquet records the codec per column chunk, so files written with different codecs can share a bucket.
`_manifest.json`	A JSON index Omnia uses to look up a session by ID and to list the dates that hold data. It is not Parquet.

Parquet schema

Every file has this schema. Columns marked optional are null when the session has no value.

Column	Type	Notes
`id`	`string`	Session ID.
`agent_name`	`string`	Agent that owned the session.
`namespace`	`string`	Kubernetes namespace of the agent.
`workspace_name`	`string`, optional	Workspace.
`virtual_user_id`	`string`, optional	Pseudonymous subject the session was recorded against.
`status`	`string`	Session status.
`created_at`	`int64`	Nanoseconds since the Unix epoch.
`updated_at`	`int64`	Nanoseconds since the Unix epoch.
`expires_at`	`int64`	Nanoseconds since the Unix epoch. `0` when unset.
`ended_at`	`int64`	Nanoseconds since the Unix epoch. `0` when unset.
`message_count`	`int32`
`tool_call_count`	`int32`
`total_input_tokens`	`int64`
`total_output_tokens`	`int64`
`estimated_cost_usd`	`float64`
`tags`	`string`	JSON array of strings.
`state`	`string`	JSON object of string values.
`last_message_preview`	`string`, optional
`messages_json`	`string`	JSON array of the session’s messages.

Each element of messages_json has the keys id, role, content and timestamp. It adds metadata, inputTokens, outputTokens, costUsd, toolCallId, sequenceNum, hasMedia and mediaTypes when they are set.

What the archive omits

The archive holds no rows from the tool_calls, provider_calls, runtime_events, runtime_event_batches and eval_results tables. The compaction job deletes those rows with the warm copy of the session.

Session retention: how a policy turns into archived and deleted sessions
SessionRetentionPolicy CRD: warm and cold retention fields
Query the session archive: SQL examples for each engine