AgentRuntime CRD

The AgentRuntime custom resource defines an AI agent deployment in Kubernetes.

API version

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: AgentRuntime

Short names: agent, ar (e.g. kubectl get agent).

Spec fields

mode

Selects the runtime shape. Defaults to agent when unset, preserving the pre-Functions behaviour. See Define Functions for when to pick which.

Value	Behaviour
agent	Long-lived conversational runtime. Default. Allows only websocket, a2a and custom facades.
function	One-shot HTTP runtime served over POST /functions/{name}. Requires inputSchema + outputSchema. Allows only rest and mcp facades, and requires exactly one rest facade.

spec:
  mode: function

inputSchema

JSON Schema (Draft 2020-12 unless the schema’s $schema says otherwise) the function’s request body is validated against. Required when mode: function; forbidden when mode: agent (CEL-gated). Stored as a raw JSON object — the facade compiles it with santhosh-tekuri/jsonschema/v6 at startup and validates every request; the runtime is schema-agnostic.

spec:
  inputSchema:
    type: object
    required: ["text"]
    properties:
      text:
        type: string

outputSchema

JSON Schema the function’s response is validated against before being returned to the caller. On output-validation failure the facade returns HTTP 502 output_invalid with the raw model output embedded so the pack author can debug schema drift (no in-runtime retry). Required when mode: function; forbidden when mode: agent (CEL-gated).

spec:
  outputSchema:
    type: object
    required: ["summary"]
    properties:
      summary:
        type: string

outputFormat

Controls how the runtime constrains the model’s response in function mode. Optional; forbidden when mode: agent (CEL-gated). When unset on a function-mode runtime it defaults to json_schema.

Value	Behaviour
text	Free-form generation; the facade still validates the output against outputSchema (HTTP 502 on mismatch).
json	Provider JSON mode — valid JSON object, shape not enforced.
json_schema	Provider structured output bound to outputSchema (default).

The runtime sets the provider response format via PromptKit. If the configured provider/model cannot honour the requested format, the invocation fails (no silent fallback) — set outputFormat: text (or json), or use a provider that supports structured outputs.

spec:
  mode: function
  outputFormat: json_schema

promptPackRef

Reference to the PromptPack containing agent prompts.

Field	Type	Required
promptPackRef.name	string	Yes
promptPackRef.version	string	No
promptPackRef.track	stable | prerelease	No

name is the pack’s logical spec.packName, not a PromptPack object name: the operator lists the PromptPacks labelled with that pack name in the agent’s namespace and picks one. version pins an exact version, compared as a string with build metadata included, so 1.0.0 does not match a pack at 1.0.0+gAAAA. track follows a channel — stable selects the highest non-prerelease version, prerelease the highest version overall. version and track are mutually exclusive (CEL-validated); with neither set, the agent follows the stable channel.

spec:
  promptPackRef:
    name: my-prompts
    version: "1.0.0"

When no PromptPack matches, PromptPackReady is False with reason PromptPackNotFound and the agent’s phase is Failed. A resolved pack that failed source or schema validation also fails the agent (PromptPackInvalid).

A pack name that has a live PromptPack resolves to it, and only a bare ref (no version, no track) may name it. A version or track on such a ref, or a spec.rollout whose stable pack or rollout.candidate.promptPackRef is live, sets PromptPackReady False with reason InvalidPromptPackRef and the phase to Failed. The validating webhook, when enabled, rejects the same specs at admission. A pack name with both a live pack and published versions resolves for no ref: PromptPackReady is False with reason PromptPackNameConflict.

An agent on a live pack runs its runtime in compile mode. The operator mounts the project read-only from the workspace content volume and gives the runtime a writable directory for the compiled pack. The facade tags every session kind:dev and arena-project:<id>, with the project id as the pack name. The pods carry no pack version or digest, and the agent records no composition. If the project can’t be mounted (the operator has no workspace content storage, or the workspace content claim workspace-<namespace>-content does not exist yet), DeploymentReady is False with reason AwaitingPodTemplateInputs, and the message names the project or the missing claim. No Deployment is created while held back; the controller watches the claim and re-reconciles as soon as it appears, so a Workspace whose first arena use is a dev agent starts once the claim is created rather than sitting Pending on PersistentVolumeClaimNotFound.

providers

A list of named provider references. Each entry maps a logical name to a Provider CRD. This enables centralized credential management, consistent configuration across agents, and explicit judge provider mapping for evals.

Field	Type	Required
providers[].name	string (pattern ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$, unique)	Yes
providers[].providerRef.name	string	Yes
providers[].providerRef.namespace	string	No. If set, must equal the AgentRuntime’s own namespace; any other value is refused (ProviderCrossNamespace)
providers[].role	Provider role	No (default llm)
providers[].requiredCapabilities	[]capability	No

role is the role the referenced Provider must have: a Provider whose spec.role differs sets ProviderReady=False (reason RoleMismatch) and puts the agent in phase Failed. requiredCapabilities lists capabilities the Provider must advertise; any missing sets ProviderReady=False (reason CapabilityMismatch) and holds the agent Pending. A Provider that is not Ready also holds the agent Pending.

Every entry other than the default is wired into the runtime by its role: embedding, tts, stt, image or inference. A platform-hosted embedding Provider (spec.platform set), such as Azure OpenAI with workload identity, works as a non-default entry the same way it works as the default: the runtime authenticates it through the cloud platform with the Provider’s spec.auth. The Provider CRD admits spec.platform only on the llm and embedding roles.

The name field is a logical identifier for looking up providers by role:

Name	Purpose
default	Primary LLM provider for the runtime
judge	LLM judge for eval execution
Any custom name	Referenced by name in PromptPack eval definitions

spec:
  providers:
    - name: default
      providerRef:
        name: claude-sonnet
    - name: judge
      providerRef:
        name: claude-haiku

See the Provider reference for details on configuring Provider CRDs (types, secrets, defaults, etc.).

framework

Agent framework configuration. Specifies which runtime framework the agent uses.

Field	Type	Default	Required
framework.type	string	promptkit	No
framework.image	string	-	No

spec:
  framework:
    type: promptkit
    image: myregistry.io/omnia-runtime:v1.0.0  # Optional image override

Framework types

Type	Description
promptkit	Default framework using PromptKit — the only framework with a built-in image
langchain	A LangChain runtime image you supply; it must implement the omnia.runtime.v1 contract
custom	Any image you supply that implements the omnia.runtime.v1 gRPC contract

Image override

The runtime image is resolved in this order: framework.image; the operator’s --framework-image=<type>=<image> entry for the type; the built-in image (promptkit only). A langchain or custom agent with neither of the first two blocks with FrameworkReady=False, reason FrameworkImageUnavailable — the operator never substitutes the PromptKit image. See the facade ↔ runtime protocol for the contract a non-PromptKit image must serve.

facades

A list of facade entries (1 to 4, no two of the same type). Each entry exposes the runtime over one protocol. An agent can expose several protocols at once by listing multiple entries — for example a websocket surface for browsers plus an a2a surface for agent-to-agent invocation, or a rest surface plus an mcp surface for a function.

Field	Type	Default	Required
facades[].type	string (websocket | a2a | rest | mcp | custom)	websocket	Yes
facades[].port	integer (1–65535)	8080	No
facades[].handler	string	runtime	No
facades[].image	string	-	No (required for type: custom)
facades[].extraEnv	[]EnvVar	-	No
facades[].drainTimeout	duration string	30s	No
facades[].clientToolTimeout	duration	60s	No
facades[].managementPlane	boolean	true	No
facades[].dataPlane	boolean	true	No
facades[].a2a	object	-	No (only for type: a2a)
facades[].mcp	object	-	No (only for type: mcp)
facades[].async	object	-	No (function mode)
facades[].expose	object	-	No

Each entry’s type must match how the runtime is invoked (CEL-validated). mode: agent runtimes use websocket, a2a and/or custom; mode: function runtimes serve HTTP at POST /functions/{name} and must have exactly one rest facade, optionally alongside an mcp facade to expose the function as an MCP tool.

handler, drainTimeout and clientToolTimeout are read from the primary facade — websocket if present, else rest, else a2a. drainTimeout is how long the facade keeps serving active realtime calls after SIGTERM before tearing the rest down; new calls stop immediately. clientToolTimeout is the maximum wait for a client tool’s result per turn. A call that is not answered in time is denied, and the turn continues. extraEnv adds environment variables to the facade container (e.g. LOG_LEVEL=debug). Add LOG_SAMPLING=off to keep every log line. By default the log keeps 1 in 100 repeats of a message per second, so under load a line can be missing for an event that happened.

The facade cancels a turn when the runtime sends nothing for 120 seconds (runtime stream inactivity timeout). The runtime sends a heartbeat every 30 seconds while a text turn runs, so a long server-side tool call, such as a sandbox cold start, does not trip it; only a runtime that has stopped answering does. To change the window, set OMNIA_RUNTIME_STREAM_INACTIVITY_TIMEOUT (a duration, at least the default 120s) in the primary facade’s extraEnv. A value that does not parse or is below 120s stops the facade from starting.

A function-mode facade also reads two bounds on Accept: text/event-stream invocations from extraEnv. OMNIA_FUNCTION_STREAM_MAX_DURATION (a duration, default 270s, at most 290s) is the longest one stream may run; at the limit the run is cancelled and the stream ends with a timeout error event. OMNIA_FUNCTION_STREAM_MAX_CONCURRENT (a positive integer, default 32) is how many streams the facade serves at once; over it a request gets 429. A value that does not parse or is out of range stops the facade from starting. See Define functions.

extraEnv is valid only on the primary facade (CEL-validated). Every facade in the pod runs in one facade container, so there is one environment to set, and the API server rejects the field on an a2a entry beside websocket, or on an mcp entry.

expose is per facade. Each facade has its own Service port (facade 8080, a2a 9999, mcp 9998 by default), and each facade that sets expose.enabled gets its own operator-managed HTTPRoute to that port: <agent>-facade for the primary facade on <agent>.<namespace>.<baseDomain>, and <agent>-<type> for a secondary one on <agent>-<type>.<namespace>.<baseDomain>. Set expose.host on either to override. Every route matches the root path, so two of an agent’s facades must not share a host: the Gateway would serve only the older route.

OMNIA_PACK_MODE, OMNIA_PACK_SOURCE_DIR, OMNIA_PACK_CONFIG_FILE, OMNIA_PACK_OUT_DIR, OMNIA_SESSION_TAGS, OMNIA_SESSION_PACK_NAME and OMNIA_IDLE_TIMEOUT are reserved. They select compile mode, stamp dev-session identity on session records, or switch on idle renewal (see Idle reaping), and only a controller sets them.

• When the AgentRuntime admission webhook is enabled, it rejects an AgentRuntime that names one of these keys in facades[].extraEnv, runtime.extraEnv or podOverrides.extraEnv. On an update it rejects only a key that was not already there.
• Whether or not the webhook runs, the operator drops these entries from the pod, and the AgentRuntime’s ReservedEnvClean condition names the key.
• A value for one of these keys from a podOverrides.extraEnvFrom ConfigMap or Secret is overridden. The operator sets OMNIA_PACK_MODE=static on the runtime container and empty OMNIA_SESSION_* values on the facade container as explicit env entries, and Kubernetes prefers those over envFrom.

type: custom is a bring-your-own facade container: it speaks the facade↔runtime gRPC contract inward and any protocol outward. It is a long-lived connection surface, valid only in mode: agent (like websocket), and requires facades[].image to point at your container. Custom facades are an enterprise-licensed capability, enforced at admission.

A custom facade must be the only entry in spec.facades (CEL-validated). It replaces the built-in facade container, so beside a websocket or a2a entry its image would never run. A custom container receives neither drainTimeout nor clientToolTimeout. A custom primary’s drainTimeout still sets the pod’s terminationGracePeriodSeconds: the drain timeout (30s when unset, at most 600s) plus 30s.

spec:
  facades:
    - type: websocket
      port: 8080
      handler: runtime
      image: myregistry.io/omnia-facade:v1.0.0  # Optional override

Handler modes

Mode	Description	Calls the runtime
runtime	Production mode using the runtime framework	Yes
demo	Demo mode with simulated streaming responses	No
echo	Simple echo handler for testing connectivity	No

facades[].managementPlane

Controls whether this facade gets an internal management-plane twin listener: the one the dashboard’s “Try this agent” view and other in-cluster control-plane callers dial. Defaults to true: the operator allocates an internal port (18080 for the WebSocket / function surface, 19999 for A2A, 19998 for MCP), publishes it under status.managementEndpoints, and the facade serves a management-plane-only auth chain there. Set false for an external-only facade: no internal listener, no *-mgmt Service port, no status entry. The external listener is unaffected either way — it runs only the data-plane validators from spec.externalAuth and never accepts management-plane tokens. Because the flag is per facade, you can keep the management plane on one surface (e.g. the WebSocket debug view) while isolating another (e.g. an external A2A endpoint).

spec:
  facades:
    - type: websocket
      managementPlane: true    # dashboard debug view works
    - type: a2a
      managementPlane: false   # external A2A surface, data-plane only
      a2a:
        taskTTL: "1h"

A facade with managementPlane: false and no data-plane validator configured can accept no callers at all; the ExternalAuth condition reports it.

facades[].dataPlane

Controls whether this facade gets its external, data-plane listener at all: the ordinary client-facing port (facade / a2a / mcp Service port) that spec.externalAuth and facades[].expose govern. Defaults to true. Set false for a facade that must be reachable only over the management-plane twin: an internal-only agent used solely from the dashboard (a dev/authoring agent is the motivating case, but this is a general option; any agent can set it). No external Service port is created for this facade, the facade container’s external port is omitted, and the facade process never binds it. Readiness and liveness are unaffected: they run on the separate health port, not the data-plane one.

dataPlane: false requires managementPlane to not be false on the same facade (CEL-validated) — a facade with both planes off would be reachable by nothing at all. facades[].expose.enabled is ignored when dataPlane is false: with no external Service port to route to, exposure would point an HTTPRoute at a backend port that does not exist, so the operator never creates the route.

This is a per-facade field, not a primary-facade one: on a dual-protocol agent (websocket + a2a) or a function agent (rest + mcp), each facade’s own dataPlane governs only that facade’s own external port. Disabling it on a secondary a2a or mcp facade removes only that facade’s a2a / mcp Service port and container port, and stops the facade process binding only that listener; the primary facade’s data plane is unaffected.

spec:
  facades:
    - type: websocket
      dataPlane: false          # management-plane-only; managementPlane stays true (default)

spec:
  facades:
    - type: websocket           # stays externally reachable
    - type: a2a
      dataPlane: false          # a2a is management-plane-only; websocket is untouched
      a2a: {}

facades[].a2a

A2A (agent-to-agent) configuration for a type: a2a facade entry; the entry’s presence is what turns A2A on.

Field	Type	Default	Description
port	integer	9999	Listener port when a websocket facade is also present. When a2a is the only agent-mode facade it is the primary listener, on the facade’s own port (default 8080)
agentCard	object	name only	Agent Card served without authentication at /.well-known/agent-card.json and /.well-known/agent.json: name (required), description, version, organization, skills[] (id, name, description, tags, examples), defaultInputModes, defaultOutputModes. capabilities is ignored: the card always advertises streaming and never push notifications. Its securitySchemes come from spec.externalAuth (clientKey, oidc, edgeJWT), and it is served in the A2A version the caller asks for (1.0 or 0.3)
taskTTL	duration string	1h	How long completed/failed/canceled tasks are retained
conversationTTL	duration string	30m	Deprecated and ignored: the A2A facade holds no conversations
taskStore.type	memory | redis	memory	Task persistence backend. Use redis when the agent runs more than one replica: in memory, tasks/get, tasks/cancel and tasks/subscribe work only on the replica that created the task, and the A2ATaskStoreShared condition turns False. With Redis, a cancel stops the turn on whichever replica runs it and a subscriber on any replica sees the task’s updates. An unreachable Redis fails facade startup rather than falling back to memory
taskStore.redisURL	string	-	Redis URL for type: redis
taskStore.redisSecretRef.name	string	-	Secret whose url key holds the Redis URL; takes precedence over redisURL
clients[]	list	-	Deprecated: use spec.a2aClients, which takes the same entries and does not need an a2a facade. Still honoured while spec.a2aClients is unset, with an admission warning; setting both is refused

spec:
  facades:
    - type: websocket
    - type: a2a
      a2a:
        port: 9999
        taskTTL: "1h"

facades[].mcp

MCP (Model Context Protocol) configuration for exposing a function as a typed MCP tool. Present only on a type: mcp facade entry, listed alongside the rest facade of a function-mode runtime; the entry’s presence is what turns the MCP server on, and it listens on mcp.port (default 9998). See Expose Functions as MCP Tools.

spec:
  facades:
    - type: rest
      port: 8080
    - type: mcp
      mcp:
        port: 9998   # defaults to 9998

facades[].async

Fire-and-forget invocation for a function: POST /functions/{name}?mode=async answers 202 with an invocation_id, and GET /functions/{name}/invocations/{id} returns its state and output. The work goes on a durable Redis-backed queue and is run by any replica of the agent.

Field	Type	Default	Description
async.enabled	boolean	false	Turns on the async routes
async.workers	integer (1–64)	1	Invocations each pod runs concurrently (per pod, so the real ceiling is workers × replicas)

Async requires a queue backend: the operator’s Redis, from the chart’s redis.default as url, host or existingSecret. With existingSecret, the Secret sits in the install namespace, so the operator copies its URL into agent-<name>-async-redis (key url) in the agent’s namespace. That Secret is owned by the AgentRuntime, is rewritten when the source changes (rolling the pods), and is deleted when async is turned off. If the source Secret or its key is missing, the condition is False with reason QueueSecretUnavailable. Without any backend the AsyncInvocationReady condition is False (reason QueueBackendMissing) and the async routes answer 501; the operator never falls back to an in-process queue, which would let two replicas run the same invocation. Without async.enabled the async routes also answer 501. An invocation whose worker dies mid-run ends as interrupted rather than being retried.

facades[].expose

Opts the agent into operator-provisioned external exposure.

Field	Type	Default	Description
expose.enabled	boolean	false	Create a host-based HTTPRoute targeting this agent’s facade Service
expose.host	string	{name}.{namespace}.{baseDomain}	Hostname override; must be one the Gateway’s listener accepts

The route is created only when the platform has a default-exposure Gateway configured (Helm defaultExposure); otherwise enabled does nothing. The resulting URLs appear under status.facade.endpoints. Exposure adds no authentication — spec.externalAuth is still the gate.

toolRegistryRef

Optional reference to a ToolRegistry resource.

Field	Type	Required
toolRegistryRef.name	string	Yes (when toolRegistryRef is set)
toolRegistryRef.namespace	string	No

Cross-namespace references are not supported: namespace must be omitted or equal the AgentRuntime’s own namespace. The agent pod’s Role is namespace-scoped, so a foreign registry would be unreadable from the pod. An AgentRuntime naming another namespace gets ToolRegistryReady=False with reason ToolRegistryCrossNamespace and is held Pending.

spec:
  toolRegistryRef:
    name: agent-tools

a2aClients

Other A2A agents this agent may call. Calling agents is the runtime’s work, so any agent can have clients — a WebSocket- or REST-only agent included; it does not need an a2a facade of its own. Replaces the deprecated spec.facades[].a2a.clients: an agent that sets both is refused (spec.a2aClients and the deprecated spec.facades[].a2a.clients are mutually exclusive), so one list is always the whole list.

Field	Type	Default	Description
name	string	required	Unique within the list (at most 32 clients)
agentRuntimeRef.name	string	-	An AgentRuntime in the same namespace; resolved to its A2A endpoint. It must have an a2a facade, or the client is unresolved
agentRuntimeRef.namespace	string	own	Only the agent’s own namespace; any other is refused (CrossNamespaceRef)
url	string	-	An external agent’s base URL, used instead of agentRuntimeRef
exposeAsTools	boolean	false	Each skill on the agent’s card becomes an a2a__<agent>__<skill> tool, offered without being declared in the pack. Each call is decided by the policy broker under the registry a2a, so on an Enterprise agent a ToolPolicy with selector.registry: a2a must permit it
authentication.secretRef.name	string	-	Secret whose token key is the bearer token sent to the agent. Mounted into the runtime as a file and read on every call, so rotating the Secret takes effect without a restart once the kubelet syncs it (typically within a minute). A missing Secret does not hold the pod: each call to the agent fails until it exists. An empty token fails each call
timeout	duration	10s / 60s	Bounds fetching the agent’s card when a conversation opens, and each call. A remote turn longer than this fails the tool call while the remote agent runs on

Consecutive calls from one conversation to one agent continue one remote conversation: the runtime sends the contextId the agent answered the first call with, and runs those calls one at a time. That memory is per runtime pod, in memory, and dropped when the conversation ends, after 50 minutes idle or when a call fails; losing it only means the next call starts a new remote conversation. A remembered contextId the agent refuses as expired is dropped and the call resent without it. Two clients that resolve to the same endpoint are refused; the second is reported unresolved. Calls use the runtime’s HTTPS_PROXY and NO_PROXY, carry headers a ToolPolicy injects, and fail on a response body over 10 MiB. A call to an agentRuntimeRef target also carries a delegation token (X-Omnia-Delegation) naming the conversation’s end user and the calling agent, requested on every call for the target’s facade audience agent-facade:<namespace>/<target>. A url target is never sent one. An a2a facade verifies an inbound token against the operator’s JWKS and refuses one that fails with 401; see Call other agents.

Resolution is reported in status.a2a.clients[] and the A2AClientsResolved condition. An unresolved client is left out of the runtime, and the agent starts without it.

spec:
  facades:
    - type: websocket
  a2aClients:
    - name: billing
      agentRuntimeRef:
        name: billing-agent
      exposeAsTools: true
      authentication:
        secretRef:
          name: billing-agent-key
      timeout: 2m

sandbox

Optional. Says whether this runtime may hold codegen sandboxes — each a network-isolated pod exposing the full codegen tool surface (shell included) over MCP, so an agent can author and check project files without reaching the rest of the cluster by default. (A SandboxClass may open egress to specific in-cluster Services; see Sandbox security.)

It does not create one. Sandboxes come from SandboxClaim resources, which the operator creates when the runtime asks for a sandbox; this block is the envelope every claim naming this runtime is checked against. What each sandbox is (its class, instance count, allocation and sync-back target) is set on the ToolRegistry’s spec.sandboxes. The operator sets OMNIA_SANDBOX_API_URL on the runtime container when enabled is true, and owns every claim it creates to this AgentRuntime, so deleting the AgentRuntime deletes its claims and sandboxes without syncing their work back.

Field	Type	Required	Default
sandbox.enabled	bool	No	false
sandbox.maxConcurrent	int (min 1)	No	4
sandbox.maxPerSession	int (min 1)	No	1
sandbox.idleTimeout	duration	No	30m
sandbox.releaseGrace	duration	No	2m
sandbox.mountReleaseGrace	duration	No	5m
sandbox.connections	[]{sandbox, role, connection or unbind}	No	none

spec:
  sandbox:
    enabled: true
    maxConcurrent: 4
    maxPerSession: 1
    idleTimeout: 30m
    releaseGrace: 2m
    mountReleaseGrace: 5m

A claim naming a runtime with enabled unset is failed, not queued.

maxConcurrent caps bound claims across all of the runtime’s sessions; maxPerSession caps them within one session. A claim past either queues (status.phase: Queued) and binds when a slot frees, rather than failing. When the runtime asks for a new sandbox and the session already holds maxPerSession live claims, of any class, the operator refuses the request (MaxPerSession) instead of creating a claim that would queue. That holds across operator replicas: a claim two replicas let through at once is queued by the controller with reason SessionLimitReached, and the request that created it deletes it and is refused the same way (see SandboxClaim).

maxPerSession caps the session’s total across sandboxes. Each ToolRegistry sandbox’s maxInstances caps that sandbox’s instances within the session.

idleTimeout is how long a bound claim may go untouched before it is deleted.

releaseGrace is how long a conversation’s sandboxes outlive the conversation. The grace starts when the conversation’s WebSocket connection closes and its last turn has finished; the operator then marks the session’s claims to be released after it. A new message in the session, an acquire or a sandbox tool call within the grace keeps them, so a client that reconnects finds the sandbox it left. Conversations over the REST and A2A facades have no connection to close, so nothing starts the grace and their sandboxes go at idleTimeout. A sandbox that mounts a project or the session workspace uses mountReleaseGrace instead. Unset is 2m; the value must be greater than zero. The idle reaper counts from the claim’s last renewal, which is never later than the conversation’s end, so with a grace at or above idleTimeout the idle reaper always deletes the claim first. See Release at conversation end.

mountReleaseGrace is releaseGrace for a sandbox that mounts a project or the session workspace. Its files live on the workspace volume, so releasing it costs the pod’s own state (caches, anything written outside the mount) and a cold start: the session’s next tool call gets a new sandbox on the same files. It is longer than releaseGrace because a person comes back to such a sandbox. Unset is 5m; the value must be greater than zero.

connections binds a Connection to one role of one sandbox in this agent’s ToolRegistry, keyed by sandbox and role. Each entry overrides the sandbox’s default for that role, so agents that share a ToolRegistry can reach the same service as different accounts. A role bound by neither is left unbound. An entry with unbind: true in place of connection drops the sandbox’s default for that role, so this agent does not reach the service at all. Each entry sets exactly one of connection and unbind: true. The SandboxConnectionsValid condition reports, before any session starts, a binding that names a sandbox the registry does not declare (ConnectionSandboxUnknown), an unbind of a required role (ConnectionRequiredUnbound), a required role that ends up unbound, or any other check that fails.

spec:
  sandbox:
    enabled: true
    connections:
      - sandbox: workspace
        role: vcs
        connection: github-acme
      - sandbox: workspace
        role: demo_sharepoint_account
        unbind: true

Each sandbox lives on the workspace content PVC workspace-<namespace>-content. When it does not exist yet and the namespace’s Workspace has storage enabled, the operator creates it and the claim waits in Pending (reason WaitingForStorage). A namespace that will never have one (no Workspace, storage disabled, or a direct-NFS install) fails the claim with reason NoWorkspaceStorage rather than leaving a pod stuck. Each sandbox’s filesystem lives on it at <workspace>/<namespace>/sandboxes/<claim>/ and survives the pod; the generated claim name in that path keeps concurrent sandboxes from sharing a filesystem.

See Sandbox security for what a sandbox can and cannot reach, and why.

context

Runtime context store configuration (working LLM context across turns).

Field	Type	Default	Required
context.type	string	memory	No
context.ttl	duration	24h	No
context.storeRef.name	string	-	Yes for redis (CEL-validated)

storeRef names a Secret that must hold the Redis connection string under the key url. A durable (redis) store lets a fresh pod resume a conversation’s working context.

spec:
  context:
    type: redis
    ttl: 24h
    storeRef:
      name: redis-credentials

Context store types:

• memory - In-memory (not recommended for production)
• redis - Redis backend (recommended)

media

Media configuration: mock:// URL resolution for the mock provider, and the media-storage backend for uploads and attachment references.

Field	Type	Default	Required
media.basePath	string	/etc/omnia/media	No
media.storage	object	-	No

spec:
  media:
    basePath: /etc/omnia/media

The runtime reads basePath from the AgentRuntime and resolves mock:// URLs against it — primarily used with the mock provider for testing multimodal responses. When media.storage is absent, the facade also uses basePath as a local media storage path. media.storage.type: none disables media storage even though basePath has a default.

media.storage

Selects the media-storage backend the operator provisions into the facade (uploads) and runtime (attachment-reference resolution) containers, as OMNIA_MEDIA_STORAGE_* environment variables. Unset or type: none disables media storage.

Field	Type	Description
type	none | local | s3 | gcs | azure	Backend (default none). Each non-none type requires its matching block (CEL-validated)
local.basePath	string	Directory the facade serves media from and the runtime resolves against; the mount path when volumeClaim is set
local.volumeClaim	string	Existing PVC mounted at basePath in both facade and runtime. Use ReadWriteMany; without it the local backend needs a writable basePath from elsewhere, since the containers have a read-only root filesystem
s3.bucket / region / prefix / endpoint	string	S3 or S3-compatible storage. bucket and region are required (CEL-validated); endpoint (e.g. MinIO) forces path-style addressing
gcs.bucket / prefix	string	Google Cloud Storage via workload identity
azure.account / container / prefix	string	Azure Blob Storage
defaultTTL	duration	Retention TTL for stored media (default 24h)
uploadURLTTL / downloadURLTTL	duration	Bounds on presigned upload / download URLs (defaults 15m / 1h)
maxFileSizeBytes	int64 (min 1)	Cap on a single stored object (default 104857600, 100 MiB)
secretRef.name	string	Explicit credentials: AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY for S3, AZURE_ACCOUNT_KEY for Azure. Omit for keyless workload identity; GCS uses workload identity only

spec:
  media:
    storage:
      type: s3
      s3:
        bucket: agent-media
        region: us-east-1
      maxFileSizeBytes: 10485760

rateLimit

A ceiling on how often this agent may be invoked. Optional; omitted means no limit, which is what every agent has unless it says otherwise.

Field	Type	Default	Required
rateLimit.maxInvocationsPerMinute	int32	— (unlimited)	No
rateLimit.burst	int32	one minute’s worth	No

spec:
  rateLimit:
    maxInvocationsPerMinute: 60
    burst: 20

An invocation over the limit is refused before it reaches the runtime, so it is never billed. How the refusal reaches the caller:

Surface	Refusal
REST / function (POST /functions/{name})	429 with Retry-After, body {"error":"rate_limited"}
MCP	tool error rate_limited: … (MCP reports tool failures at HTTP 200)
WebSocket / A2A	the turn ends with the rate-limit error
Async (?mode=async)	not refused — queued items wait, see below

The refusal is not 502 runtime_error: the runtime was never invoked, so nothing ran and the call works later.

Refusals are also counted in status.invocations as failed invocations with the limit as lastError, so a runaway loop is visible in kubectl get agentruntime rather than only in logs. The facade logs one line per 30s while throttling, with a count of the suppressed refusals.

A queued invocation waits rather than being refused: the async path does not retry, so a 202-accepted item is paced at the ceiling instead of failing. The wait is bounded at 30s; past that it is refused like anything else. A shutdown while waiting is reported as a cancellation, not as the rate limit.

The facade reads the ceiling from the CRD at startup, so it is part of the Deployment’s config hash; changing the limit rolls the pod.

• The limit is per pod, not per agent: each facade process holds its own bucket, so an agent with three replicas admits up to three times this rate.
• It counts invocations, not spend: a ceiling on frequency, not a currency budget.
• burst absorbs a batch that fans out at once and then goes quiet; unset, it defaults to one minute’s worth.

It applies to every facade an agent serves (websocket, a2a, rest, mcp) from one shared bucket: the limit is on the agent, not on a protocol.

runtime

Deployment-related settings including replicas, resources, and autoscaling.

Field	Type	Default	Description
runtime.replicas	int32 (min 0)	1	Desired pods; ignored when autoscaling is enabled
runtime.resources	ResourceRequirements	requests 100m CPU, 256Mi memory	Runtime container resources. Replaces the default. The facade (100m, 128Mi) and policy-broker (50m, 64Mi) containers get fixed requests
runtime.nodeSelector / tolerations / affinity	-	-	Pod scheduling
runtime.volumes / volumeMounts	-	-	See below
runtime.extraEnv	[]EnvVar	-	Extra environment variables for the runtime container (e.g. LOG_LEVEL=debug; LOG_SAMPLING=off keeps every log line under load)
runtime.events.record	all | lifecycle | none	all	Which pipeline telemetry events are recorded. See below
runtime.autoscaling	object	-	See below

spec:
  runtime:
    replicas: 3
    resources:
      requests:
        cpu: "500m"
        memory: "256Mi"
      limits:
        cpu: "1000m"
        memory: "512Mi"
    nodeSelector:
      node-type: agents
    tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "agents"
        effect: "NoSchedule"

runtime.events

runtime.events.record selects how much pipeline telemetry the runtime records for a session: the runtime events a session’s trace shows. A session’s messages, tool calls, provider calls and eval results are always recorded; whether a session is recorded at all is decided by its service group having a session-api, not by this field.

Value	Records
all (default)	Every pipeline event
lifecycle	Everything except the per-stage and per-check events that succeed: stage.started, stage.completed, prompt.template.started, prompt.template.rendered, validation.started, validation.passed. Failures are kept
none	None of PromptKit’s pipeline telemetry (pipeline.*, stage.*, middleware.*, validation.*, prompt.template.*, context.*, state.*, workflow.*)

At every level Omnia’s own events are recorded: ToolPolicy decisions and tool-call receipts (which become audit-log rows), the client-tool hand-off, and stream interruptions.

Telemetry carries no user content but is most of what a turn writes: about 50 events for a three-turn conversation, more than half of them stage.*. Converting it is a measurable share of the runtime’s CPU under load. An event left out costs nothing.

spec:
  runtime:
    events:
      record: lifecycle

runtime.volumes and runtime.volumeMounts

Mount additional volumes in the runtime container for media files, mock configurations, or other data.

Field	Type	Description
runtime.volumes	[]Volume	Kubernetes Volume definitions
runtime.volumeMounts	[]VolumeMount	Volume mounts for the runtime container

spec:
  runtime:
    volumes:
      - name: mock-media
        persistentVolumeClaim:
          claimName: media-pvc
      - name: mock-config
        configMap:
          name: mock-responses
    volumeMounts:
      - name: mock-media
        mountPath: /etc/omnia/media
        readOnly: true
      - name: mock-config
        mountPath: /etc/omnia/mock
        readOnly: true

Supported volume types include:

• persistentVolumeClaim - Mount a PVC for persistent storage
• configMap - Mount a ConfigMap as files
• secret - Mount a Secret as files
• emptyDir - Temporary storage (cleared on pod restart)

runtime.autoscaling

Horizontal pod autoscaling configuration. Supports both standard HPA and KEDA.

Field	Type	Default	Description
enabled	boolean	false	Enable autoscaling
type	string	hpa	hpa or keda
minReplicas	integer	1	Minimum replicas. 0 is KEDA scale-to-zero and requires keda.triggers (CEL-validated)
maxReplicas	integer	100	Maximum replicas
targetMemoryUtilizationPercentage	integer	70	Memory target (HPA only)
targetCPUUtilizationPercentage	integer	90	CPU target (HPA only)
scaleDownStabilizationSeconds	integer	300	Scale-down cooldown (HPA only)

When an agent sets no runtime.autoscaling, it inherits the autoscaling default declared on its Workspace service group, if any. The AutoscalingReady condition reports which applies: True/Scaling when an HPA or KEDA ScaledObject is reconciled, True/Disabled for static replicas, and False/KEDANotInstalled when type: keda is requested but the KEDA CRDs are absent (the agent then stays at static replicas).

Standard HPA example

spec:
  runtime:
    autoscaling:
      enabled: true
      type: hpa
      minReplicas: 2
      maxReplicas: 10
      targetMemoryUtilizationPercentage: 70
      targetCPUUtilizationPercentage: 80
      scaleDownStabilizationSeconds: 300

KEDA example

spec:
  runtime:
    autoscaling:
      enabled: true
      type: keda
      minReplicas: 1  # 0 for scale-to-zero, which requires keda.triggers
      maxReplicas: 20
      keda:
        pollingInterval: 15
        cooldownPeriod: 60
        triggers:
          - type: prometheus
            metadata:
              serverAddress: "http://prometheus-server:9090"
              query: 'sum(omnia_agent_connections_active{agent="my-agent"})'
              threshold: "10"

runtime.autoscaling.keda

KEDA-specific configuration (only used when type: keda).

Field	Type	Default	Description
pollingInterval	integer (min 1)	30	Seconds between trigger checks
cooldownPeriod	integer (min 0)	300	Seconds before scaling down
triggers	array	-	Custom KEDA triggers (type, metadata)
connectionThreshold	integer (min 1)	1000	Target active connections per pod for the default trigger; only used when triggers is empty. Half the facade’s per-pod cap of 2000 connections, which it answers with 503 past. Set lower (e.g. 20) for audio/media workloads

If no triggers are specified, a default Prometheus trigger scales on sum(omnia_agent_connections_active{agent="<name>",namespace="<namespace>"}) or vector(0) against connectionThreshold. Its serverAddress is fixed at http://omnia-prometheus-server.omnia-system.svc.cluster.local/prometheus, so set triggers explicitly when Prometheus runs elsewhere.

For type: hpa the operator also sets scaling behaviour that has no field: scale-up has a 0-second stabilisation window and may add 100% of pods or 4 pods every 15 seconds (whichever is larger); scale-down uses scaleDownStabilizationSeconds and removes at most 50% of pods per 60 seconds.

KEDA trigger types

Prometheus trigger:

triggers:
  - type: prometheus
    metadata:
      serverAddress: "http://prometheus:9090"
      query: 'sum(rate(requests_total[1m]))'
      threshold: "100"

Cron trigger:

triggers:
  - type: cron
    metadata:
      timezone: "America/New_York"
      start: "0 8 * * 1-5"   # 8am weekdays
      end: "0 18 * * 1-5"    # 6pm weekdays
      desiredReplicas: "5"

evals

Configures realtime eval execution for this agent. When enabled, session events trigger evaluation of live conversations against eval definitions in the referenced PromptPack. See Realtime Evals for the full architecture and Configure Realtime Evals for a step-by-step guide.

Field	Type	Default	Required
evals.enabled	boolean	false	No

spec:
  evals:
    enabled: true

evals.inline and evals.worker

Split realtime evals between two paths by group. Each eval’s groups are auto-classified by handler type, or declared explicitly on the eval, and decide which path runs it.

Field	Type	Default	Description
evals.inline.groups	string[]	["fast-running"]	Groups executed inside the runtime process, in the PromptKit SDK’s eval middleware
evals.worker.groups	string[]	["long-running", "external"]	Groups executed out-of-band in the eval-worker, which reads session events from Redis streams

spec:
  evals:
    enabled: true
    inline:
      groups: ["fast-running"]
    worker:
      groups: ["long-running", "external"]

The defaults are disjoint — no eval runs on both paths. An absent or empty list falls back to the default; to disable all evals, use enabled: false (an empty groups list is not an off-switch).

Rows written to the eval_results table are tagged source="runtime-inline" from the inline path and source="worker" from the eval-worker, so aggregation over the table can distinguish them.

Judge provider resolution

LLM judge evals resolve their provider from the AgentRuntime’s spec.providers list, using the entry named judge (or a custom name referenced in the PromptPack’s eval definitions). See providers for the reference shape.

The eval worker resolves provider credentials from the referenced Provider CRDs and their associated Secrets.

Fixed eval behaviour

The following hold for every AgentRuntime:

• Sampling: no spec.evals field sets a sampling rate. Every session is evaluated except where a PromptPack eval trigger samples it (sample_turns, sample_sessions with sample_percentage; see the PromptPack reference).
• Rate limiting: the eval-worker applies no throttle to eval executions or concurrent judge calls.
• Session completion: the eval-worker treats a session as complete, and runs its on_session_complete evals, after 5 minutes with no new message (DefaultInactivityTimeout in ee/pkg/evals/completion_tracker.go).
• Inline execution: in the runtime, the SDK runs each turn’s inline evals in a background goroutine, so they do not delay the reply. At most 10 run at once (DefaultMaxConcurrentEvals); a turn dispatched while all 10 slots are busy has its inline evals skipped. Inline on_session_complete evals run when the conversation closes.

evals.podOverrides

Customises the eval-worker Pod (see Configure pod overrides). The eval-worker is per (namespace, service group) pair, not per AgentRuntime: a namespace with multiple eval-enabled service groups gets multiple eval-workers, one per group. Reconciliation deterministically picks the first AgentRuntime per group, regardless of which one triggered the reconcile, and a WorkspaceServiceGroup’s own evalWorker.podOverrides takes precedence over this field.

memory

Configures cross-session memory for this agent. Memory is disabled unless enabled: true. The memory-api endpoint is resolved from the Workspace service group selected by spec.serviceGroup.

Field	Type	Default	Description
enabled	bool	false	Enables memory capability
retrieval.enabled	bool	true when memory is enabled	Enables ambient per-turn retrieval, for a pack that renders {{memory_context}}
retrieval.strategy	enum	unset (keyword)	keyword, semantic, or composite. semantic and composite fall back to keyword search when the memory store has no semantic capability
retrieval.limit	int32	10	Maximum episodic memories injected per turn; 1–50
retrieval.accessFilter.denyCEL	string	empty	CEL deny expression over result metadata
tools.enabled	bool	true when memory is enabled	Enables explicit memory tools
tools.notes	string	empty	Advice for this agent, appended to every memory tool’s description under “Notes for this agent:”; at most 2000 characters
facet	string	empty	Pins the agent’s memory to one container (a project, client or matter). 1 to 128 ASCII letters, digits and . _ : / -, starting with a letter or digit

Ambient retrieval and explicit tools are independent. Set retrieval.enabled: false to keep explicit tools without automatic context injection, or set tools.enabled: false for read-only ambient retrieval. Ambient retrieval also needs the pack’s prompt to render {{memory_context}}; a pack without it is not retrieved for.

tools.notes adds to the workspace’s memory tool guidance, which memory-api renders from the MemoryPolicy; it never replaces it. Use it when agents in one workspace need different advice, such as which identifier to use as about_key. Advice that applies to every agent in the workspace belongs in the MemoryPolicy’s toolGuidance.

facet fixes the container every conversation of the agent saves into and recalls from, on every facade. Without it, a WebSocket client chooses the container with the memory_facet connection parameter, which isolates relevance but does not restrict the user: the same user can connect without it. With it, a connection that names no container uses the agent’s, and one that names another is refused with 403. See Memory tools.

The runtime reads spec.memory when it starts, so the operator rolls the agent’s pods whenever spec.memory changes. An edit takes effect once the new pods are ready.

Memory tools are registered only for conversations with a user identity. An anonymous conversation therefore has neither explicit memory tools nor a user-scoped memory write, even when this configuration is enabled.

spec:
  serviceGroup: default
  memory:
    enabled: true
    retrieval:
      strategy: composite
      limit: 10
      accessFilter:
        denyCEL: 'metadata.classification == "restricted"'
    tools:
      enabled: true
      notes: "Use the customer's account id as about_key for profile facts."

See Memory tools and the Memory API reference.

rollout

Progressive rollout configuration. When rollout.candidate is set and differs from the current spec, the controller creates a candidate Deployment and progresses through the defined steps.

Field	Type	Required	Description
rollout.candidate	object	No	Sparse overrides for the candidate version. Unset means no rollout is active
rollout.candidate.promptPackRef	object	No	PromptPack for the candidate (name, version or track, as in spec.promptPackRef)
rollout.candidate.providerRefs	array	No	Provider list for the candidate (same shape as spec.providers)
rollout.candidate.toolRegistryRef	object	No	ToolRegistry override for the candidate
rollout.steps	array (min 1)	Yes	Ordered sequence of rollout actions; set exactly one field per step
rollout.steps[].setWeight	integer	—	Set candidate traffic weight (0-100)
rollout.steps[].pause	object	—	Pause the rollout
rollout.steps[].pause.duration	string	No	Pause duration (e.g., “5m”). Omit to wait for manual promotion
rollout.steps[].analysis	object	—	Run a RolloutAnalysis template; the rollout advances only if it passes
rollout.steps[].analysis.templateName	string	Yes	Name of the RolloutAnalysis in the agent’s namespace
rollout.steps[].analysis.args	array	No	name/value argument overrides for the template
rollout.stickySession	object	No	Consistent routing for experiments; applied only with trafficRouting.istio
rollout.stickySession.hashOn	string	Yes	Header for consistent hashing (e.g., “x-user-id”)
rollout.rollback	object	No	Rollback configuration
rollout.rollback.mode	string	No	automatic, manual (default), or disabled. automatic rolls back when an analysis step fails or the candidate reports ProgressDeadlineExceeded. manual never rolls back on its own. disabled behaves exactly like manual: the controller does not distinguish them
rollout.rollback.progressDeadlineSeconds	integer (min 10)	No	Sets the candidate Deployment’s spec.progressDeadlineSeconds. Automatic rollback on an unhealthy candidate fires when that deadline is exceeded (ProgressDeadlineExceeded), so this bounds detection time. Unset inherits the Kubernetes default (600s)
rollout.trafficRouting	object	No	How the step’s weight is delivered
rollout.trafficRouting.mode	string	No	mesh, replicaWeighted or external. Unset resolves to mesh when the mesh is available, else replicaWeighted; a config that sets istio without mode resolves to external
rollout.trafficRouting.mesh.hosts	array	No	Hosts the operator-owned VirtualService matches; defaults to the agent’s own Service DNS name
rollout.trafficRouting.mesh.stableSubset / candidateSubset	string	No	DestinationRule subset names (default stable / canary)
rollout.trafficRouting.mesh.waypoint	string	No	Istio ambient waypoint; the operator labels the agent Service istio.io/use-waypoint=<waypoint> so the L7 split takes effect
rollout.trafficRouting.istio.virtualService.name	string	Yes	Existing VirtualService to patch (mode: external)
rollout.trafficRouting.istio.virtualService.routes	array (min 1)	Yes	Route names to manage
rollout.trafficRouting.istio.destinationRule.name	string	Yes	Existing DestinationRule to patch
rollout.trafficRouting.istio.destinationRule.stableSubset / candidateSubset	string	No	Subset names (default stable / canary)
rollout.trigger.promptPackChannel	stable | prerelease	Yes (when trigger is set)	Version-triggered canary: when a version newer than the agent’s active one appears on this channel, the controller sets it as rollout.candidate. Requires a version-pinned spec.promptPackRef and is mutually exclusive with promptPackRef.track (CEL-validated)

replicaWeighted approximates the weight by the ratio of candidate to stable replicas; status.rollout.trafficWeightEnforced is false in that mode, and the TrafficRouting condition reports when the resolved mode degraded from the one requested.

Enterprise

The analysis step type requires the RolloutAnalysis CRD, which is an enterprise feature. See the RolloutAnalysis CRD.

Rollout example

# Canary rollout with analysis
spec:
  promptPackRef:
    name: customer-support-pack
    version: "1.0.0"
  rollout:
    candidate:
      promptPackRef:
        name: customer-support-pack
        version: "2.0.0"
    steps:
      - setWeight: 10
      - pause:
          duration: "5m"
      - analysis:
          templateName: quality-check
      - setWeight: 50
      - pause:
          duration: "10m"
      - setWeight: 100
    rollback:
      mode: automatic
    trafficRouting:
      istio:
        virtualService:
          name: customer-support-vs
          routes: [primary]
        destinationRule:
          name: customer-support-dr

When candidate matches the current spec, the rollout is idle. Promotion copies the candidate overrides into the main spec, keeps 100% of traffic on the candidate until the stable Deployment is healthy on the new config, then cuts back to stable, deletes the candidate Deployment and clears rollout.candidate. Rollback reverts the candidate to match the current spec.

externalAuth

Data-plane authentication for the facades’ external listeners. Each sub-block enables one validator. The external chain runs the enabled validators in the order clientKeys, oidc, edgeTrust. With no sub-block the chain is empty and the external listener returns 401 to every request. For setup, see Configure agent authentication.

Field	Type	Default	Description
clientKeys	object	-	Enables per-caller client keys. An empty object enables the validator
clientKeys.defaultRole	string	viewer	Value of the role claim for a key whose Secret carries no scopes. Any string is valid
clientKeys.trustEndUserHeader	bool	false	Takes the end user from the X-End-User-Id request header. When false, the end user is the key’s own identity
oidc	object	-	Verifies RS256 JWTs from your identity provider
oidc.issuer	string	-	Required. Must equal the token’s iss claim exactly. The controller fetches {issuer}/.well-known/openid-configuration after trimming trailing slashes
oidc.audience	string	-	Required. The expected aud claim
oidc.claimMapping.subject	string	sub	Claim used as the caller’s subject
oidc.claimMapping.endUser	string	sub	Claim used as the end user. Falls back to the subject when the token lacks it
oidc.sessionClaim	string	-	Claim that binds a token to one conversation. A token without it is rejected with 401 (missing_session_claim). A session the token starts on the WebSocket or A2A facade is bound to the claim’s value, and the token can resume only a session bound to the same value. The check fails closed, so a resume needs session-api, a session shed as unrecorded cannot be resumed, and an A2A client starts a conversation by sending no contextId. Function invocations are never resumed
edgeTrust	object	-	Trusts claim headers that an upstream edge has already verified
edgeTrust.headerMapping.subject	string	x-user-id	Header read as the subject. The request is admitted only when this header is present
edgeTrust.headerMapping.endUser	string	x-user-id	Header read as the end user. Falls back to the subject
edgeTrust.headerMapping.email	string	x-user-email	Header read into the email claim
edgeTrust.claimsFromHeaders	map	-	Inbound header name (case-insensitive) to the claim name it is exposed as

The edgeTrust role header is always x-user-roles. It is read into the role claim, and the claim is viewer when the header is absent. It is not configurable. Every other claim of an admitted request, including the IdP’s own role and group claims for oidc, is forwarded under its own name and reaches ToolPolicy as principal.claims.

The controller mirrors the issuer’s JWKS into the Secret agent-<agentruntime>-oidc-jwks and refreshes it every six hours. The OIDCJWKSReady condition reports the result, and the facade does not start with oidc enabled until that Secret exists.

A WebSocket connection admitted by an oidc token closes with code 4401 shortly before the token’s exp. The client can keep it open by sending a new token in a credential_refresh frame. See WebSocket protocol.

Client key Secret

Each client key is a Secret in the agent’s namespace. hack/mint-client-key.sh writes one; any tool that applies a Secret can write the same shape. The facade watches the Secrets, so a created or deleted Secret takes effect without a change to the AgentRuntime.

Part	Value
metadata.name	agent-<agent>-clientkey-<id>. The part after -clientkey- is the key’s subject. A name that does not match uses the full Secret name
metadata.labels	omnia.altairalabs.ai/credential-kind: agent-client-key and omnia.altairalabs.ai/agent: <agent>
metadata.ownerReferences	An entry naming the AgentRuntime’s UID. The facade skips a Secret without one and logs it
data.keyHash	The raw 32-byte SHA-256 digest of the token, not its hex string. The facade skips any other length and logs it at debug level
data.scopes	Optional JSON string map. Becomes the key’s claims
data.expiresAt	Optional RFC 3339 timestamp. Without it the key never expires

Other spec fields

Field	Type	Description
externalAuth	object	Data-plane authentication for the facades’ external listeners. Unset means the external listener rejects every request, and only the management plane admits callers. See externalAuth
console	object	Dashboard console settings: allowedAttachmentTypes (client-side default: image/png, image/jpeg, image/gif, image/webp, audio/mpeg, audio/wav, audio/ogg, application/pdf, text/plain, text/markdown, text/javascript, text/typescript, application/javascript, application/typescript, text/x-python, application/x-python-code, text/csv, application/json), allowedExtensions, maxFileSize (bytes on the wire after base64, default 10485760), maxFiles (1–20, default 5), mediaRequirements
duplex	object	Realtime voice. enabled renders the dashboard’s voice call console and makes the agent require the duplex_audio and interruption runtime capabilities; audio (recommendedSampleRate, channels, format, chunkDurationMs) is the format the runtime counter-offers in RuntimeHello; mode (realtime or cascaded) selects whether audio streams to the model or runs through the agent’s stt and tts providers. Unset mode selects cascaded when the default provider’s type cannot stream audio (claude, ollama, vllm) and both speech providers are usable, and realtime otherwise. cascaded without both speech providers fails each session at open. See Add voice to a text-only agent
serviceGroup	string	Workspace service group (spec.services[].name) whose session-api and memory-api the agent uses. Default default; DNS-label pattern, max 63
privacyPolicyRef.name	string	A SessionPrivacyPolicy in the agent’s namespace that overrides the service group’s policy for this agent; reported by the PrivacyPolicyResolved condition. See Configure privacy policies
extraPodAnnotations	map	Extra annotations on the agent pods
podOverrides	object	Pod-level scheduling, service account, labels, annotations and volumes, plus container-level extraEnv / extraEnvFrom / extraVolumeMounts applied to the facade and runtime containers (not to operator-injected sidecars). See podOverrides

podOverrides

PodOverrides is one object type with six placements. Every field is optional.

CRD	Field	Pod
AgentRuntime	spec.podOverrides	Facade and runtime
AgentRuntime	spec.evals.podOverrides	Eval-worker for the agent’s service group
Workspace	spec.services[].evalWorker.podOverrides	Eval-worker for the group; takes precedence over spec.evals.podOverrides
Workspace	spec.services[].session.podOverrides	Managed session-api
Workspace	spec.services[].memory.podOverrides	Managed memory-api
ArenaJob	spec.workers.podOverrides	Worker Job pods

Field	Type	Merge
serviceAccountName	string	Replaces the operator default when non-empty
priorityClassName	string	Replaces the operator default when non-empty
runtimeClassName	string	Sets spec.runtimeClassName on the pod when non-empty. Empty leaves the cluster default. DNS-subdomain pattern, max 253. The operator does not check that the RuntimeClass exists. See Run agents under gVisor or Kata
labels	map	Merged. Operator-set keys win on collision
annotations	map	Merged. Your values win on collision
nodeSelector	map	Merged by key. Your values win on collision
tolerations	[]Toleration	Appended
imagePullSecrets	[]LocalObjectReference	Appended
extraVolumes	[]Volume	Appended to the pod’s volumes. The schema does not validate the volume; the API server does when the Deployment is applied
extraEnv	[]EnvVar	Appended to the env of each container
extraEnvFrom	[]EnvFromSource	Appended to the envFrom of each container
extraVolumeMounts	[]VolumeMount	Appended to the volumeMounts of each container

The container-level fields (extraEnv, extraEnvFrom, extraVolumeMounts) apply to every container in the pod except operator-injected sidecars. On an AgentRuntime pod that means the facade and runtime containers and not policy-broker. AgentRuntime drops reserved environment keys from extraEnv.

PodOverrides has no resources, securityContext, affinity, topologySpreadConstraints, hostNetwork, hostPID or hostIPC field. An AgentRuntime sets affinity with runtime.affinity.

On an AgentRuntime pod:

• runtime.nodeSelector and runtime.tolerations apply first. podOverrides then merges key-wise into the node selector and appends to the tolerations.
• facades[].extraEnv and runtime.extraEnv target one container each.
• podOverrides.annotations override extraPodAnnotations on key collision.
• The pod’s ServiceAccount is podOverrides.serviceAccountName, else the Workspace’s spec.runtime.serviceAccountName, else <agentruntime>-facade. An AgentRuntime that sets its own serviceAccountName takes no Workspace podLabels or podAnnotations defaults.
• The pod’s RuntimeClass is podOverrides.runtimeClassName, else the Workspace’s spec.runtime.runtimeClassName, else the cluster default. It applies to every container in the pod, policy-broker included. An agent that sets its own serviceAccountName still inherits the Workspace runtimeClassName.

For session-api and memory-api, RBAC bindings follow serviceAccountName.

See Configure pod overrides for worked examples.

Idle reaping

The omnia.altairalabs.ai/idle-timeout annotation makes an agent temporary. Its value is a Go duration such as 30m or 2h. Once the agent goes that long without use, the operator deletes it, and garbage collection removes everything it owns.

metadata:
  annotations:
    omnia.altairalabs.ai/idle-timeout: 2h

• Use is any invocation through any facade surface. The facade records it in status.lastUsedAt, at most once per half the timeout, so the recorded time lags the last use by up to half the timeout.
• The deadline is lastUsedAt plus the timeout. An agent that has never been used is judged from its creation, so an agent whose pods never start is still deleted. A lastUsedAt in the future counts as the current time.
• The effective idle window is between half the timeout and the full timeout after the last use, because renewals are throttled to one per half the timeout.
• When the deadline passes, the operator deletes the AgentRuntime and records a Normal event with reason IdleReaped.
• An invalid value sets the IdleTimeoutValid condition to False with reason InvalidIdleTimeout and a Warning event. The agent is not deleted.
• Without the annotation an agent is never deleted for being idle, and its facade never writes lastUsedAt.

The facade’s Role reaches only its own AgentRuntime: get on agentruntimes and get/patch on agentruntimes/status, both limited by resourceNames to this agent. A pod cannot change another agent’s lastUsedAt.

Hidden resources

The omnia.altairalabs.ai/hidden label controls visibility, for any Omnia resource kind, not only AgentRuntimes. An object labelled "true" is left out of lists and pickers. It is still reconciled, and it can still be reached by name.

metadata:
  labels:
    omnia.altairalabs.ai/hidden: "true"

These surfaces leave hidden objects out:

• Every dashboard list of workspace and shared resources: agents, PromptPacks, ToolRegistries, ToolPolicies, Providers and skill sources. That includes every page and picker built on them, such as the agents page, topology, the console agent picker, the session filters, the Arena run dialog, the deploy profile and the quality page’s agent picker.
• The dashboard’s workspace stats, and its credential Secrets list.

Hidden means “not listed”, never “not reachable”. Any URL that names a hidden object shows it. That covers the agent page /agents/<name>, the console’s ?agent=<name> link, and the pack and registry pages. A get by name is never filtered. A workspace list returns hidden objects only when asked with ?includeHidden=true, which the dashboard uses only to resolve names. For example, a hidden registry’s page uses it to show its hidden users.

Hidden is a visibility setting for lists and UI, never a compliance exemption, so audit, compliance and DSAR surfaces such as the privacy API’s agent register still include hidden objects. Nor is it an accounting exemption: a hidden agent’s cost is still tracked, and it keeps its own row on the costs page, and usage telemetry counts hidden objects like any other.

The deploy API labels every object of an Arena project’s dev intent this way, and a project PromptPackSource keeps its live PromptPack’s label in step with its own. See deploy intent. The label says only whether an object is shown. What the object is, such as omnia.altairalabs.ai/dev, is a separate label, and no list filters on it. A dev intent’s objects carry omnia.altairalabs.ai/dev: "true", omnia.altairalabs.ai/arena-project (the project id) and omnia.altairalabs.ai/arena-user (the user pseudonym).

Status fields

phase

Value	Description
Pending	No ready replica yet, or a reference is not usable yet (Provider not Ready, missing required capabilities, cross-namespace ToolRegistry)
Running	At least one replica is ready
Failed	A configuration error that will not resolve on its own: no PromptPack matches promptPackRef, the PromptPack failed validation, a referenced Provider does not exist, or a Provider’s role does not match providers[].role

replicas

Field	Description
status.replicas.desired	Desired replicas
status.replicas.ready	Ready replicas
status.replicas.available	Available replicas

Other status fields

Field	Description
status.activeVersion	The deployed PromptPack version
status.compositionDigest	sha256:<hex> over everything resolved for the agent — pack content, skill manifest, tool registry and providers. Empty until every input has published its own identity
status.serviceEndpoint	Internal facade endpoint, {name}.{namespace}.svc.cluster.local:{port}
status.managementEndpoints	Internal management-plane ports (ws, a2a, mcp), one per surface whose facade has managementPlane enabled
status.a2a	A2A agentCardURL and endpoint (only with an a2a facade), and per-client resolution of spec.a2aClients (clients[]: name, resolvedURL, ready, error) on any agent with clients
status.facade.endpoints	External URLs derived from observed Gateway API HTTPRoutes (protocol, url, scheme, host, path, port, routeName, routeNamespace, valid, reason). Empty means in-cluster only
status.runtimeCapabilities	Contract capabilities the running runtime advertises
status.invocations	Facade-reported total, succeeded, lastInvocationTime, lastSuccessTime and lastError. Throttled and approximate — a pod that dies loses up to one 30-second flush interval of counts
status.lastUsedAt	When the agent was last used, written by its facade. Only an agent with the omnia.altairalabs.ai/idle-timeout annotation has it; see Idle reaping
status.observedGeneration	Last generation the controller observed

rollout (status)

Field	Description
status.rollout.active	Whether a rollout is in progress
status.rollout.currentStep	Current step index (zero-based)
status.rollout.currentWeight	Current candidate traffic weight
status.rollout.stableVersion	Version serving stable traffic
status.rollout.candidateVersion	Version serving candidate traffic
status.rollout.startedAt / stepStartedAt	When the rollout, and the current step, began
status.rollout.message	Human-readable rollout state
status.rollout.trafficRoutingMode	The traffic-routing mode actually in use
status.rollout.trafficWeightEnforced	false when the weight is approximated by replica ratio
status.rollout.analysis	Progress of the analysis step being executed
status.rollout.promoting	A promotion is in progress

conditions

Type	Description
Ready	True when at least one replica is ready
DeploymentReady	Deployment is ready
ServiceReady	Service is ready
PromptPackReady	Referenced PromptPack resolved and is valid
ProviderReady	Referenced Providers exist, are Ready, match the declared role and advertise the required capabilities
ToolRegistryReady	Referenced ToolRegistry exists and is in the agent’s namespace. A missing registry sets it False and the agent runs without tools
FrameworkReady	A runtime image resolved for framework.type
CapabilitiesSatisfied	The running runtime advertises every capability the spec requires; while False for the current generation the Deployment is scaled to zero
RuntimeCapabilitiesReported	The runtime has reported its capabilities
ExternalAuth	Effective facade authentication; False when a facade can accept no callers
OIDCJWKSReady	The OIDC issuer’s JWKS is mirrored (when externalAuth.oidc is set)
AutoscalingReady	Which autoscaling policy is in effect (see runtime.autoscaling)
AsyncInvocationReady	Async invocation can run (see facades[].async)
Volumes	Every PersistentVolumeClaim the pod mounts exists
SessionRecording	The agent’s service group has a session-api, so conversations are recorded. False with WorkspaceNotFound, ServiceGroupNotFound or NoSessionService when it does not; the agent still serves, but writes no history. Never gates Ready
PrivacyPolicyResolved	Which privacy policy applies
WithinWorkspaceQuota	The workspace’s spec.quotas.agents applied to this agent. Present only when the Workspace sets an agent quota. True with QuotaSatisfied, or RunningOverAgentRuntimeQuota for an agent admitted before maxAgentRuntimes was lowered beneath it. False with ReplicasClamped when it asks for more replicas than maxReplicasPerAgent: it runs at the cap and is otherwise reconciled as usual. False with AgentRuntimeQuotaExceeded when the Workspace has not admitted it and it has no Deployment: nothing is created for it, phase Pending, Ready reason WorkspaceQuotaExceeded, until it is admitted
IdleTimeoutValid	Whether the omnia.altairalabs.ai/idle-timeout annotation is a valid duration. Present only on an annotated agent
RolloutActive / TrafficRouting	Rollout progress and traffic-routing health
A2AClientsResolved	Every spec.a2aClients entry resolved; False names the unresolved ones (reason CrossNamespaceRef or Unresolved). Present only on an agent with clients
A2ATaskStoreShared	A2A tasks are visible to every replica (see facades[].a2a.taskStore). Present only on an agent with an a2a facade

Printed columns

kubectl get agentruntime shows PHASE, READY (ready replicas), VERSION (status.activeVersion), INVOCATIONS, LAST-SUCCESS and AGE.

Complete example

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: AgentRuntime
metadata:
  name: production-agent
  namespace: agents
spec:
  promptPackRef:
    name: customer-service-prompts
    version: "2.1.0"

  providers:
    - name: default
      providerRef:
        name: claude-production
    - name: judge
      providerRef:
        name: claude-haiku

  toolRegistryRef:
    name: service-tools

  facades:
    - type: websocket
      port: 8080
      handler: runtime

  context:
    type: redis
    ttl: 24h
    storeRef:
      name: redis-credentials

  evals:
    enabled: true

  runtime:
    replicas: 3  # Ignored when autoscaling enabled
    resources:
      requests:
        cpu: "500m"
        memory: "256Mi"
      limits:
        cpu: "1000m"
        memory: "512Mi"
    autoscaling:
      enabled: true
      type: keda
      minReplicas: 1
      maxReplicas: 20
      keda:
        pollingInterval: 15
        cooldownPeriod: 120
        triggers:
          - type: prometheus
            metadata:
              serverAddress: "http://omnia-prometheus-server.omnia-system.svc.cluster.local/prometheus"
              query: 'sum(omnia_agent_connections_active{agent="production-agent",namespace="agents"}) or vector(0)'
              threshold: "10"