# Scale agent deployments

Scale an `AgentRuntime` to a fixed replica count, or let an HPA or KEDA adjust it with load.

## Prerequisites

- A running `AgentRuntime`. The examples call it `my-agent`.
- For HPA, the Kubernetes Metrics Server.
- For KEDA, KEDA installed in the cluster, and Prometheus if you scale on connections. See [Autoscaling](/explanation/agents/autoscaling) for how the operator wires both.

## Set a fixed replica count

Set `spec.runtime.replicas`:

```yaml
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: AgentRuntime
metadata:
  name: my-agent
spec:
  runtime:
    replicas: 3
  # ...
```

Or patch it:

```bash
kubectl patch agentruntime my-agent --type=merge \
  -p '{"spec":{"runtime":{"replicas":5}}}'
```

When autoscaling is enabled, the autoscaler owns the replica count and `runtime.replicas` has no effect.

## Choose an autoscaler

Use HPA when CPU and memory track your load and you want no extra dependencies. Use KEDA for connection-based scaling, cron schedules, external events or scale-to-zero.

Keep `minReplicas: 1` for interactive agents. A scaled-to-zero agent refuses the first connection, and the default connections trigger cannot wake it, so the API server rejects a KEDA policy with `minReplicas: 0` and no `keda.triggers`. To scale to zero, list a trigger that does not depend on the agent's pods, such as `cron`.

## Autoscale with HPA

Enable the HPA:

```yaml
spec:
  runtime:
    autoscaling:
      enabled: true
      type: hpa
      minReplicas: 2
      maxReplicas: 10
      targetMemoryUtilizationPercentage: 70
      targetCPUUtilizationPercentage: 90
```

`minReplicas` defaults to 1, `maxReplicas` to 100, the memory target to 70 and the CPU target to 90. The CPU target cannot be turned off.

HPA utilisation targets are percentages of the container's resource requests. See [Set resource requests](#set-resource-requests).

## Autoscale with KEDA

Set `type: keda`:

```yaml
spec:
  runtime:
    autoscaling:
      enabled: true
      type: keda
      minReplicas: 1
      maxReplicas: 20
```

With no `triggers`, the operator adds a Prometheus trigger on `omnia_agent_connections_active` for the agent, with a threshold of `keda.connectionThreshold` connections per pod (default 1000, half the facade's per-pod cap of 2000). The trigger queries the Prometheus server that the Omnia chart installs. Lower `connectionThreshold` for audio and media agents.

If your Prometheus runs elsewhere, or you want other signals, define `keda.triggers` yourself. The `serverAddress` below is illustrative:

```yaml
spec:
  runtime:
    autoscaling:
      enabled: true
      type: keda
      minReplicas: 1
      maxReplicas: 20
      keda:
        pollingInterval: 30
        cooldownPeriod: 300
        triggers:
          - type: prometheus
            metadata:
              serverAddress: "http://prometheus:9090"
              query: 'sum(omnia_agent_connections_active{agent="my-agent",namespace="default"}) or vector(0)'
              threshold: "10"
```

The `or vector(0)` makes the query return 0 when no series exist.

A cron trigger raises the replica count during business hours:

```yaml
triggers:
  - type: cron
    metadata:
      timezone: "America/New_York"
      start: "0 8 * * 1-5"   # 8am Mon-Fri
      end: "0 18 * * 1-5"    # 6pm Mon-Fri
      desiredReplicas: "5"
```

Metrics to scale on, all labelled `agent` and `namespace`:

| Metric | Meaning | Scale on it |
|--------|---------|-------------|
| `omnia_agent_connections_active` | Current WebSocket connections | Yes, a direct load indicator |
| `omnia_agent_requests_inflight` | Requests being processed (pending LLM calls) | Yes, shows processing load |
| `omnia_agent_sessions_active` | Active sessions | Depends on session lifetime |
| `omnia_agent_request_duration_seconds` | Request latency | No, it trails the load |

See [Set Up Observability](/how-to/observability/setup-observability) for the full metric list.

## Tune autoscaling

For a latency-sensitive agent, lower the memory target and keep pods longer (HPA):

```yaml
autoscaling:
  type: hpa
  targetMemoryUtilizationPercentage: 50
  scaleDownStabilizationSeconds: 600
```

For a cost-sensitive agent, allow more connections per pod (KEDA):

```yaml
autoscaling:
  type: keda
  minReplicas: 1
  keda:
    connectionThreshold: 400
```

For predictable traffic, combine a cron floor with connection-based scaling (KEDA). Add the Prometheus trigger as well, because defining any trigger replaces the default one:

```yaml
autoscaling:
  type: keda
  keda:
    triggers:
      - type: cron
        metadata:
          timezone: "UTC"
          start: "0 8 * * 1-5"
          end: "0 18 * * 1-5"
          desiredReplicas: "5"
      - type: prometheus
        metadata:
          serverAddress: "http://prometheus:9090"
          query: 'sum(omnia_agent_connections_active{agent="my-agent",namespace="default"}) or vector(0)'
          threshold: "10"
```

## Set resource requests

Every container in the agent pod has CPU and memory requests, so an HPA utilisation target can compute. The operator sets these defaults, as requests with no limits:

| Container | CPU | Memory |
|-----------|-----|--------|
| `facade` | `100m` | `128Mi` |
| `runtime` | `100m` | `256Mi` |
| `policy-broker` | `50m` | `64Mi` |

On a small cluster running many mostly idle agents, scale every default CPU request at once with the operator flag `--agent-cpu-request-percent` (default `100`). For example, set `operator.extraArgs: ["--agent-cpu-request-percent=10"]` in the chart values. Every container still gets a request, so an HPA can still compute utilisation, but utilisation is then measured against the smaller request.

`spec.runtime.resources` replaces the runtime container's defaults. The facade and policy-broker defaults cannot be changed per agent:

```yaml
spec:
  runtime:
    resources:
      requests:
        cpu: "500m"
        memory: "256Mi"
      limits:
        cpu: "1000m"
        memory: "512Mi"
```

## Run more than one replica

Use the Redis context store so any replica can resume a conversation:

```yaml
spec:
  context:
    type: redis
    storeRef:
      name: redis-credentials
```

`storeRef` names a Secret with a `url` key. The operator owns the agent's Service and resets its spec on every reconcile, so you cannot set `sessionAffinity` on it. See [Sessions](/explanation/agents/sessions) for how the memory store behaves with several replicas.

## Verify

Check replica status. The `Ready` column shows ready replicas:

```bash
kubectl get agentruntime my-agent
kubectl describe agentruntime my-agent
```

The `AutoscalingReady` condition reports whether the HPA or ScaledObject was reconciled. For HPA, inspect the HPA, which is named after the agent:

```bash
kubectl get hpa my-agent
kubectl describe hpa my-agent
```

For KEDA, inspect the ScaledObject, which is also named after the agent, and the HPA that KEDA manages:

```bash
kubectl get scaledobject my-agent
kubectl get hpa keda-hpa-my-agent
```

If `AutoscalingReady` is `False` with reason `KEDANotInstalled`, install KEDA. The agent stays at its static replica count until you do.

## Related pages

- [Autoscaling](/explanation/agents/autoscaling) explains HPA versus KEDA and what scale-to-zero costs.
- [Set Up Observability](/how-to/observability/setup-observability) covers the scaling metrics.
- [AgentRuntime reference](/reference/core/agentruntime) lists every autoscaling field.