Architecture Overview

Omnia is a Kubernetes operator plus a set of standalone services. This page explains which component owns what, how a request moves through them, and the trade-offs behind the split.

How the pieces fit

The operator turns custom resources into agent pods. The agent pod serves the live request path. Session, memory and privacy data belong to separate per-workspace services, so the facade never writes to a database.

graph TB
    clients((Clients))
    dash[Dashboard<br/>static files, nginx, port 3000]
    mgmt[Management API<br/>cmd/management-api]

    subgraph cluster["Kubernetes cluster"]
        op[Operator<br/>cmd/main.go]

        subgraph pod["Agent pod"]
            facade[Facade<br/>cmd/agent]
            runtime[Runtime<br/>cmd/runtime]
            broker[Policy Broker<br/>EE sidecar, port 8090]
        end

        session[Session API]
        memory[Memory API]
        privacy[Privacy API<br/>EE]
        pg[(PostgreSQL)]
        redis[(Redis<br/>optional hot cache)]
    end

    clients -->|WebSocket| facade
    clients --> dash
    clients -->|/api| mgmt
    dash -->|/api| mgmt
    mgmt -->|HTTP| op
    mgmt -->|management twin| facade
    mgmt -->|HTTP| session
    op -->|creates Deployment, Service, tools ConfigMap| pod
    facade -->|gRPC| runtime
    runtime -->|"POST /v1/decision"| broker
    facade -->|HTTP record| session
    runtime -->|HTTP events| session
    runtime -->|HTTP memory| memory
    session --> pg
    session --> redis
    memory --> pg
    session -->|audit drain| privacy
    memory -->|audit drain| privacy
    privacy -->|erasure fan-out| session
    privacy -->|erasure fan-out| memory

SERVICES.md at the repository root holds the full node and edge topology, the protocol table and the tracing inventory.

Operator

The operator is the controller manager in cmd/main.go. It reconciles AgentRuntime, PromptPack, ToolRegistry and Provider resources, and it serves the REST API under internal/api. For each AgentRuntime it creates a Deployment, a Service and a ConfigMap holding the tools configuration. The Reconciliation page describes the loop in detail.

Dashboard

The dashboard is a separate Deployment with its own image and port. It is Next.js’s static export served by nginx, with no server code. Every API call it makes goes to management-api (cmd/management-api): the Gateway or Ingress routes /api there, and the dashboard’s nginx proxies any /api request that reaches it.

management-api signs users in, holds the API keys, checks workspace roles and reads and writes custom resources as the caller’s Workspace role. It calls the operator’s APIs, Session API, Memory API and the agent’s management-plane twin (see below), so “Try this agent” works from the browser. It holds the only management-plane signing key and serves the JWKS that the facades and the operator verify against.

Agent pod

Each agent pod runs a facade container and a runtime container. Enterprise installs add the Policy Broker as a third container, which the operator injects.

The facade handles client protocols. It serves the agent’s surfaces, translates client messages into gRPC calls to the runtime, manages connection lifecycle (connect, disconnect, heartbeat, park and resume for realtime sessions), and records the conversation. The recording happens off the gRPC bus and goes to Session API over HTTP.

The runtime wraps the LLM provider through the PromptKit SDK and executes tools. It records turn and tool events to Session API over HTTP and calls Memory API over HTTP when memory is enabled. The facade dials the runtime over gRPC on localhost.

An AgentRuntime lists one or more single-protocol facades in spec.facades[], each with a type of websocket, a2a, rest or mcp (the schema also admits custom). One pod can expose several surfaces, each on its own port, all backed by one runtime. AgentRuntime lists the fields.

Media uploads are optional. When spec.media.storage selects a backend, the facade serves POST /media/request-upload, and cloud backends hand the client a presigned URL for a direct upload. Configure media storage covers the setup.

Plane isolation

Each management-capable facade surface listens on two ports. The external port runs the data-plane auth chain from spec.externalAuth. The internal management-plane twin accepts only management JWTs minted by management-api: facade-mgmt (18080) for WebSocket, a2a-mgmt (19999) for A2A and mcp-mgmt (19998) for MCP.

The twin ports are on a ClusterIP Service, never on an external Gateway or HTTPRoute, and the twin rejects any request without a valid management JWT. spec.facades[].managementPlane gates each twin and defaults to enabled. The operator publishes enabled twins on AgentRuntime.status.managementEndpoints, and management-api’s WebSocket proxy and Doctor read that status to dial the agent. A dashboard session therefore reaches an agent even when no external auth validators are configured.

Data services

Session, memory and privacy data live in standalone per-workspace services, so the agent pod holds no state and storage scales on its own.

Session API (cmd/session-api) stores sessions in three tiers: an optional Redis hot cache, a PostgreSQL warm store and a cold archive. The Redis cache sits inside Session API and is not a separate store.
Memory API (cmd/memory-api) stores cross-session memory in PostgreSQL with pgvector for semantic search.
Privacy API (ee/cmd/privacy-api, Enterprise) owns consent and opt-out preferences and is the central audit hub: Session API and Memory API drain their enforcement records to it. It also runs the right-to-erasure lifecycle, fanning deletion out to every service group’s Session API and Memory API.

Multi-tenancy explains how these services are scoped to a workspace.

Policy Broker

The Policy Broker (ee/cmd/policy-broker, Enterprise) is a sidecar in the agent pod. It sits outside the request path as a decision service. The runtime calls it once per tool call with POST /v1/decision on POLICY_BROKER_URL (http://localhost:8090). The broker evaluates ToolPolicy CEL rules against the request headers, body and identity, and returns allow or deny plus any headers to inject. The runtime enforces the answer. If the broker is unreachable the runtime denies the call, unless POLICY_BROKER_FAIL_MODE is set to open.

Tool calls

The next diagram follows one tool call from the client to the tool backend.

sequenceDiagram
    participant C as Client
    participant F as Facade
    participant R as Runtime
    participant B as Policy Broker
    participant T as Tool backend

    C->>F: WebSocket message
    F->>R: gRPC request
    R->>R: Send to LLM (PromptKit)
    R->>R: LLM returns tool_call
    R->>B: POST /v1/decision
    B-->>R: allow or deny, injected headers
    R->>T: HTTP, gRPC, MCP or OpenAPI call
    T-->>R: Tool result
    R->>R: Send result to LLM
    R-->>F: Stream response
    F-->>C: WebSocket chunks

The runtime’s OmniaExecutor.dispatch makes the broker call and then routes by handler type to the HTTP, MCP, gRPC or OpenAPI executor. The broker step applies only on installs that run the Policy Broker.

Tools declared with a client handler run in the browser. The runtime sends them to the facade, which forwards a tool_call message over the WebSocket. Tool execution model covers where each tool type runs.

Resource relationships

An AgentRuntime references a PromptPack, a ToolRegistry and one or more Providers by name. The operator rejects a Provider or ToolRegistry reference that names another namespace, because a Provider carries a credential and nothing is shared across workspaces.

graph LR
    AR[AgentRuntime] -->|promptPackRef| PP[PromptPack]
    AR -->|toolRegistryRef| TR[ToolRegistry]
    AR -->|providers| PR[Provider]
    AR -->|creates| D[Deployment]
    AR -->|creates| S[Service]
    AR -->|generates| CM[Tools ConfigMap]
    PP -->|configMapRef| CM2[Prompt ConfigMap]
    PR -->|secretRef| SEC[Secret]
    D -->|runs| FC[Facade container]
    D -->|runs| RC[Runtime container]

The runtime reads the Provider and its credential Secret from the Kubernetes API, from the Provider’s namespace. The operator does not mount provider credentials into the pod. A change to the PromptPack, the ToolRegistry or a Provider alters the config hash on the Deployment, so the pods roll.

Realtime evals

Eval definitions live in the PromptPack, next to validators and guardrails, and run as sessions progress. Two paths execute them, split by eval group.

Inline: the runtime evaluates during the turn through an in-process listener on the PromptKit event bus. The default group is fast-running (deterministic handlers such as contains and regex), which is cheap enough to gate on. Results are tagged source="runtime-inline".
Worker: Session API publishes events to Redis Streams, and an eval worker consumes them, loads the pack’s eval definitions and runs the assertions. The default groups are long-running and external (LLM judges and external API checks). Results are tagged source="worker".

Both write to the eval_results table. The defaults are disjoint, so an eval runs on one path unless you override the routing per agent.

The operator creates one eval worker Deployment per service group, for service groups that have eval-enabled AgentRuntimes that are not PromptKit agents. PromptKit agents evaluate inline and join the worker path only when their service group sets evalWorker.enabled. See Realtime evals for the full design and AgentRuntime for the fields.

Design decisions

Why a Kubernetes operator

Agents become Kubernetes resources, so they get declarative configuration, status conditions, RBAC and the platform’s rescheduling and scaling machinery. The cost is that the operator is a hard dependency: without it, changes to a resource do not reach the pods.

Why a facade and a runtime in one pod

The two containers have different jobs. The facade owns client protocols and connection state. The runtime owns the LLM pipeline and tool execution. The split lets you add a protocol by adding a facade type, or swap the runtime for a custom one, without touching the other side. It also lets each side be tested alone.

The costs are real. Replicas and autoscaling apply to the whole pod, so the facade and runtime always scale together even when only one is the bottleneck. Every message also crosses one extra gRPC hop on localhost.

Why WebSocket for the primary facade

LLM responses stream, and client tool calls and their results flow in both directions on the same connection. A persistent connection also serves a multi-turn conversation and lets the facade park and resume realtime sessions. The rest and mcp facades cover one-shot HTTP callers.

Why a separate PromptPack

Prompts live in their own resource, so several agents can share one pack and prompt changes are versioned apart from the deployment. A progressive rollout, configured in the AgentRuntime’s spec.rollout, can canary a new pack version. Rollout strategies describes the options. The trade-off is another resource to keep consistent with the agent that references it.

Why a handler-based ToolRegistry

A ToolRegistry holds handlers of type http, grpc, mcp, openapi, client or platform. MCP and OpenAPI handlers describe their own tools, so the registry discovers them, while HTTP and gRPC handlers declare their schemas explicitly. One registry manages all types. Editing a registry rolls the pods that reference it, so a tool change is a rollout and not a hot reload.

Reconciliation
Multi-tenancy for workspaces, roles (owner, editor, viewer) and per-workspace services
Tool execution model
Realtime evals
Set up observability for tracing and metrics
AgentRuntime, PromptPack, ToolRegistry and Provider reference