Progressive rollouts

In this tutorial we roll a new prompt version out to a running agent. We send a quarter of the traffic to the new version, then half, then all of it, and watch the controller promote it.

We use replica-weighted routing, which splits traffic by the ratio of pods and needs no service mesh.

What we will build

The AgentRuntime controller runs a second Deployment for the new version beside the stable one. Both sit behind the agent’s Service, and the controller changes the replica counts at each step.

graph TB
    AR[AgentRuntime my-assistant] -->|scales replicas| STABLE[Deployment my-assistant<br/>PromptPack 1.0.0]
    AR -->|creates, scales, deletes| CANARY[Deployment my-assistant-canary<br/>PromptPack 2.0.0]
    SVC[Service my-assistant] --> STABLE
    SVC --> CANARY

Prerequisites

• The cluster and agent from Getting started: AgentRuntime my-assistant in the default namespace, running PromptPack assistant-pack version 1.0.0. That agent’s promptPackRef names the pack without a version, so it follows the highest stable version published.
• kubectl.
• Autoscaling is not enabled on my-assistant.

Step 1: pin the agent and run four replicas

An unpinned agent moves to a new version as soon as it is published, so we pin my-assistant to 1.0.0 first. Replica-weighted routing splits traffic by pod count, so we also scale to four replicas, which is enough to express a 25% step:

kubectl patch agentruntime my-assistant -n default --type merge \
  -p '{"spec":{"promptPackRef":{"version":"1.0.0"},"runtime":{"replicas":4}}}'

The controller replaces the agent pod and adds three more. Check the agent:

kubectl get agentruntime my-assistant -n default
kubectl get deployment my-assistant -n default

The AgentRuntime’s VERSION column shows 1.0.0. When the pods are ready, the Deployment’s READY column shows 4/4.

Step 2: publish the new PromptPack version

Each version is its own PromptPack object. Both versions share spec.packName. We create version 2.0.0 with a ConfigMap and a PromptPack, saved together as assistant-pack-v2.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: assistant-prompts-v2
  namespace: default
data:
  pack.json: |
    {
      "$schema": "https://promptpack.org/schema/latest/promptpack.schema.json",
      "id": "assistant",
      "name": "Assistant",
      "version": "2.0.0",
      "template_engine": {
        "version": "v1",
        "syntax": "{{variable}}"
      },
      "prompts": {
        "main": {
          "id": "main",
          "name": "Main Assistant",
          "version": "2.0.0",
          "system_template": "You are a helpful AI assistant. Answer in two sentences or fewer and ask one clarifying question when the request is ambiguous.",
          "parameters": {
            "temperature": 0.3,
            "max_tokens": 1024
          }
        }
      }
    }
---
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: PromptPack
metadata:
  name: assistant-pack-v2
  namespace: default
spec:
  packName: assistant-pack
  version: "2.0.0"
  source:
    type: configmap
    configMapRef:
      name: assistant-prompts-v2

kubectl apply -f assistant-pack-v2.yaml
kubectl get promptpack -n default

The list shows assistant-pack-v2 at version 2.0.0 in phase Active. assistant-pack, version 1.0.0, moves to Superseded, because a newer stable version now exists.

The pin holds the agent on the old version. Check it:

kubectl get agentruntime my-assistant -n default

The VERSION column still shows 1.0.0.

Step 3: start the rollout

We describe the rollout in a patch file, rollout-patch.yaml. The candidate points at version 2.0.0 of the same pack, and the steps move traffic to 25%, 50% and 100% with a two-minute pause after each of the first two:

spec:
  rollout:
    candidate:
      promptPackRef:
        name: assistant-pack
        version: "2.0.0"
    steps:
      - setWeight: 25
      - pause:
          duration: "2m"
      - setWeight: 50
      - pause:
          duration: "2m"
      - setWeight: 100
    rollback:
      mode: automatic
    trafficRouting:
      mode: replicaWeighted

kubectl patch agentruntime my-assistant -n default --type merge \
  --patch-file rollout-patch.yaml

The controller sees that the candidate differs from spec.promptPackRef and starts the rollout. Check the status:

kubectl get agentruntime my-assistant -n default \
  -o jsonpath='{.status.rollout}' | jq

The output includes these fields:

{
  "active": true,
  "currentWeight": 25,
  "stableVersion": "1.0.0",
  "candidateVersion": "2.0.0",
  "trafficRoutingMode": "replicaWeighted",
  "trafficWeightEnforced": false
}

trafficWeightEnforced is false because the weight is approximated by the replica ratio.

Step 4: see the traffic split

The controller created a second Deployment, my-assistant-canary, for the candidate, and scaled my-assistant down to make room for it:

kubectl get deployment -n default

my-assistant shows 3 replicas and my-assistant-canary shows 1, so one pod in four, and roughly a quarter of the requests, reaches the candidate. The split holds for the whole two-minute pause. The candidate’s pods run the new pack version:

kubectl get deployment my-assistant-canary -n default \
  -o jsonpath='{.spec.template.spec.containers[?(@.name=="runtime")].env[?(@.name=="OMNIA_PROMPTPACK_VERSION")].value}{"\n"}'

The output is 2.0.0.

Step 5: watch the weight increase

After the first two-minute pause the controller sets the weight to 50%. We watch the Deployments until the replicas are even:

kubectl get deployment -n default -w

my-assistant and my-assistant-canary each show 2 replicas. After the second pause the weight reaches 100%: my-assistant-canary runs all four replicas and my-assistant drops to 0.

Step 6: confirm the promotion

When all steps finish, the controller copies the candidate into spec.promptPackRef. The candidate keeps serving all traffic while the stable Deployment rolls to the new pack. Then the controller cuts traffic back to the stable Deployment, deletes the candidate Deployment and clears rollout.candidate.

Check the result:

kubectl get agentruntime my-assistant -n default \
  -o jsonpath='{.spec.promptPackRef.version}{"\n"}'
kubectl get deployment -n default

The version is 2.0.0, and only my-assistant is left, with 4 replicas. The controller records the promotion as an event:

kubectl get events -n default \
  --field-selector involvedObject.name=my-assistant,reason=RolloutPromoted

The agent now runs the new prompt on every replica and the rollout is idle.

Next steps

• Roll back a rollout when a candidate misbehaves.
• Rollout strategies covers blue/green, experiments and sticky sessions.
• The AgentRuntime rollout reference lists every field, including Istio traffic routing.
• The RolloutAnalysis reference shows how to gate a step on metrics.