Production setup

In this tutorial we install Omnia on a fresh cluster and set it up for production use.

What you will build

An Istio service mesh for traffic management and mTLS
The Gateway API for external agent access
An observability stack (Prometheus, Grafana, Loki, Tempo)
JWT authentication for secure agent access

At the end, a client reaches a production agent through the external Gateway with a JWT, and we read its metrics in Grafana.

flowchart LR
  Client -->|wss + JWT| GW["Gateway omnia-agents"]
  GW --> Route["HTTPRoute production-agent-route"]
  Route --> Svc["Service production-agent :8080"]
  Svc --> Agent["AgentRuntime production-agent"]
  Agent --> Redis[(Redis)]
  KEDA --> Agent
  Prometheus --> KEDA
  Agent -.->|metrics| Prometheus
  Prometheus --> Grafana
  IGW["Gateway omnia-internal"] --> Grafana

Prerequisites

A fresh Kubernetes cluster (1.26+) with no Omnia installed. This tutorial installs Omnia itself, so it does not build on Getting started.
kubectl and helm v3 installed. Step 1 downloads istioctl.
Domain name with DNS configured (for external access)
A TLS certificate and key for agents.yourdomain.com
A Redis instance reachable from the cluster
KEDA not already installed in the cluster

Step 1: install Istio

We install the Gateway API CRDs, then Istio:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/standard-install.yaml

curl -L https://istio.io/downloadIstio | sh -
cd istio-*
export PATH=$PWD/bin:$PATH

istioctl install --set profile=default -y

kubectl label namespace default istio-injection=enabled

Verify Istio is running:

kubectl get pods -n istio-system

The istiod pod is Running.

Step 2: install Omnia with full stack

Create production-values.yaml:

replicaCount: 2

dashboard:
  auth:
    mode: builtin

# Builtin users live in management-api's own PostgreSQL database, and
# signed-in sessions in a Redis every management-api replica shares.
# Each Secret holds a connection URL: POSTGRES_CONN for the database, and
# `url` for Redis.
managementApi:
  database:
    existingSecret: omnia-management-db
redis:
  default:
    existingSecret:
      name: omnia-redis
      key: url

prometheus:
  enabled: true
  server:
    persistentVolume:
      enabled: true
      size: 50Gi

grafana:
  enabled: true
  adminPassword: "your-secure-password"  # Change this!
  persistence:
    enabled: true
    size: 10Gi

loki:
  enabled: true
  singleBinary:
    persistence:
      enabled: true
      size: 50Gi

alloy:
  enabled: true

tempo:
  enabled: true
  persistence:
    enabled: true
    size: 50Gi

istio:
  enabled: true

gateway:
  enabled: true
  className: istio
  listeners:
    https:
      enabled: true
      tlsSecretName: omnia-tls  # We create this secret in step 3

internalGateway:
  enabled: true
  className: istio

keda:
  enabled: true

CRDs install first, as their own release. The app chart refuses to install without them, and helm upgrade of the app chart never updates them:

helm install omnia-crds oci://ghcr.io/altairalabs/charts/omnia-crds \
  --devel \
  --namespace omnia-system --create-namespace

This installs every kind, the Enterprise ones (Arena, ToolPolicy, RolloutAnalysis, SessionPrivacyPolicy, PromptPackSource) included. Pass --set enterprise.enabled=false to leave the Enterprise kinds out.

Now install Omnia:

helm install omnia oci://ghcr.io/altairalabs/charts/omnia \
  --devel \
  -n omnia-system \
  -f production-values.yaml

Verify the release:

kubectl get pods -n omnia-system
kubectl get gateway -n omnia-system

The operator pod is Running, and the Gateways omnia-agents and omnia-internal are listed.

Step 3: configure TLS certificate

Create the TLS secret the external Gateway’s HTTPS listener refers to:

kubectl create secret tls omnia-tls \
  -n omnia-system \
  --cert=path/to/tls.crt \
  --key=path/to/tls.key

Verify the secret:

kubectl get secret omnia-tls -n omnia-system

The secret has type kubernetes.io/tls.

Step 4: deploy a production agent

We start with the PromptPack the agent runs: a ConfigMap holding the compiled pack, and the PromptPack that references it. Save both manifests in one file, promptpack.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: assistant-prompts
  namespace: default
data:
  pack.json: |
    {
      "$schema": "https://promptpack.org/schema/latest/promptpack.schema.json",
      "id": "assistant",
      "name": "Assistant",
      "version": "1.0.0",
      "template_engine": {
        "version": "v1",
        "syntax": "{{variable}}"
      },
      "prompts": {
        "main": {
          "id": "main",
          "name": "Main Assistant",
          "version": "1.0.0",
          "system_template": "You are a helpful AI assistant. Be concise and accurate in your responses. Always be polite and professional.",
          "parameters": {
            "temperature": 0.7,
            "max_tokens": 4096
          }
        }
      }
    }
---
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: PromptPack
metadata:
  name: assistant-pack
  namespace: default
spec:
  packName: assistant-pack
  version: "1.0.0"
  source:
    type: configmap
    configMapRef:
      name: assistant-prompts

Apply the file:

kubectl apply -f promptpack.yaml

configmap/assistant-prompts created
promptpack.omnia.altairalabs.ai/assistant-pack created

Verify the PromptPack:

kubectl get promptpack assistant-pack

The PHASE column shows Active.

Next, create a Secret holding the Redis connection URL for the agent’s context store. The Secret must have a url key. Put your Redis instance’s URL in place of the example:

kubectl create secret generic redis-credentials \
  -n default \
  --from-literal=url=redis://redis.example.internal:6379

The command prints secret/redis-credentials created.

Then create a Provider for your LLM, with the API key in a Secret. Save both manifests in one file, provider.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: llm-credentials
  namespace: default
stringData:
  ANTHROPIC_API_KEY: "sk-ant-..."
---
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
metadata:
  name: claude-production
  namespace: default
spec:
  type: claude
  model: claude-sonnet-4-20250514
  credential:
    secretRef:
      name: llm-credentials
  defaults:
    temperature: "0.7"
    maxTokens: 4096

Replace the placeholder key and apply the file:

kubectl apply -f provider.yaml

secret/llm-credentials created
provider.omnia.altairalabs.ai/claude-production created

Verify the Provider:

kubectl get provider claude-production

The PHASE column shows Ready.

Next, create an AgentRuntime with production settings. The externalAuth.oidc block makes the facade validate JWTs from your identity provider. See Configure Authentication for the other providers. Save the manifest as agentruntime.yaml:

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: AgentRuntime
metadata:
  name: production-agent
  namespace: default
spec:
  promptPackRef:
    name: assistant-pack
  providers:
    - name: default
      providerRef:
        name: claude-production
  facades:
    - type: websocket
      port: 8080
      handler: runtime  # Production mode
  externalAuth:
    oidc:
      issuer: "https://your-identity-provider.com"
      audience: "omnia-agents"
  context:
    type: redis
    storeRef:
      name: redis-credentials
    ttl: "24h"
  runtime:
    resources:
      requests:
        cpu: "500m"
        memory: "256Mi"
      limits:
        cpu: "1000m"
        memory: "512Mi"
    autoscaling:
      enabled: true
      type: keda
      minReplicas: 1
      maxReplicas: 10
      keda:
        pollingInterval: 15
        cooldownPeriod: 300
        connectionThreshold: 50

Replace issuer and audience with your identity provider’s values, and apply the file:

kubectl apply -f agentruntime.yaml

agentruntime.omnia.altairalabs.ai/production-agent created

Verify the agent:

kubectl get agentruntime production-agent

The PHASE column shows Running once the agent pod is ready.

Finally, create an HTTPRoute to expose the agent through the omnia-agents Gateway. The rewrite strips the /production-agent prefix, because the facade serves /ws. Save the manifest as httproute.yaml:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: production-agent-route
  namespace: default
spec:
  parentRefs:
    - name: omnia-agents
      namespace: omnia-system
  hostnames:
    - "agents.yourdomain.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /production-agent
      filters:
        - type: URLRewrite
          urlRewrite:
            path:
              type: ReplacePrefixMatch
              replacePrefixMatch: /
      backendRefs:
        - name: production-agent
          port: 8080

Replace agents.yourdomain.com with your domain, and apply the file:

kubectl apply -f httproute.yaml

httproute.gateway.networking.k8s.io/production-agent-route created

Verify that the Gateway accepted the route:

kubectl get httproute production-agent-route \
  -o jsonpath='{.status.parents[0].conditions[?(@.type=="Accepted")].status}{"\n"}'

The output is True.

Step 5: access observability tools

We port-forward to Grafana:

kubectl port-forward svc/omnia-grafana 3000:80 -n omnia-system

Open http://localhost:3000/grafana/ and sign in with:

Username: admin
Password: the grafana.adminPassword from your values file

The Omnia dashboards are already loaded:

Omnia Overview: request rate, latency, cost and token usage
Omnia Agent Detail: one agent’s requests, latency, errors, connections and logs
Omnia LLM Costs: cost and token usage by model and agent

Step 6: connect to your agent

With the external Gateway configured, we connect over WebSocket with a JWT from your identity provider:

websocat "wss://agents.yourdomain.com/production-agent/ws?agent=production-agent" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

The agent accepts the connection.

Verification checklist

Istio pods running in istio-system
Omnia operator running in omnia-system
Prometheus scraping metrics
Grafana dashboards loading
Gateway receiving external traffic
Agent pods running with Istio sidecar
WebSocket connections working through Gateway

Next steps

Set Up Observability - Configure dashboards and alerts
Configure Authentication - Detailed JWT setup
Scale Agent Deployments - KEDA and HPA configuration
Autoscaling Explained - Understanding scaling strategies