ArenaJob CRD

Enterprise Feature

ArenaJob is an enterprise feature. The Arena CRDs come from the omnia-crds chart, which installs them by default; the Arena controllers run only when enterprise.enabled=true in the app chart’s values. See Installing a License for details.

The ArenaJob custom resource is a template for test executions that run scenarios from an ArenaProject bundle. It reads the arena config file inside that bundle (selected by spec.arenaFile) and describes evaluation, load testing, or data generation work with configurable workers and output destinations.

An ArenaJob does not execute anything itself. Each execution is an ArenaRun (short name arun) that the job creates, owns and names <job-name>-<suffix>: once when the job is first seen if it has no schedule, or on every cron tick if it has one. The run carries a frozen copy of the job’s spec (spec.template), so editing the job never changes what an existing run did. Execution state (phase, progress, results, timings) lives on the ArenaRun; the ArenaJob’s status describes scheduling only. See ArenaRun below.

API Version

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob

Short name: aj (e.g. kubectl get aj).

Spec Fields

sourceRef

Reference to the ArenaProject, in the job’s namespace, whose repository holds the arena config file, scenarios and prompts. Required. A run whose project does not exist yet, or is not Ready, stays Pending and is retried every 30 seconds.

Field	Type	Required	Description
name	string	Yes	Name of the ArenaProject

spec:
  sourceRef:
    name: my-evaluation-source

ref

The project ref a run executes: a branch, a tag, a full ref name or a full commit SHA. Defaults to the project’s default branch (main for a hosted project). See Refs for what is accepted.

Each run resolves the ref to a commit SHA when it starts and records it on the ArenaRun’s status.source. A job naming a branch runs whatever the branch holds when each run starts; a job naming a SHA runs that commit every time. A ref that names nothing in the repository fails the run with reason RefNotFound, and a commit that cannot be exported (see ArenaProject snapshots) fails it with SnapshotUnexportable.

spec:
  sourceRef:
    name: my-evaluation-source
  ref: refs/tags/v1.2.0

arenaFile

Path to the arena config file within the project’s repository, matched exactly against the bundle contents. Defaults to config.arena.yaml.

It names one file: a run loads exactly one arena config. A value containing a glob character (*, ? or [), such as evals/*.arena.yaml, is rejected when the ArenaJob is applied.

See the Arena Config File reference for the schema of this file.

spec:
  arenaFile: config.arena.yaml

type

The type of job to execute. Defaults to evaluation.

Value	Description
evaluation	Run prompt evaluation against test scenarios (default)
loadtest	Run load testing against providers
datagen	Generate synthetic data using prompts

spec:
  type: evaluation

trials

Number of times to repeat each scenario × provider combination. Overrides per-scenario trials defined in the scenario YAML files. Minimum 1.

• For evaluation jobs, trials provide statistical confidence (pass rate, flakiness score).
• For loadtest jobs, trials define the total load volume, consumed under concurrency control.

spec:
  trials: 20

scenarios

Filter which scenarios to run from the arena file. If not specified, runs all scenarios defined in the arena file.

Field	Type	Description
include	[]string	Glob patterns for scenarios to include
exclude	[]string	Glob patterns for scenarios to exclude

Exclusions are applied after inclusions. If include is empty, all scenarios are included by default.

spec:
  scenarios:
    include:
      - "scenarios/critical-*.yaml"
    exclude:
      - "*-slow.yaml"

budgetLimit

The most this job may spend, in budgetCurrency, before it is stopped.

spec:
  budgetLimit: "25.00"
  budgetCurrency: "USD"

It applies to any run (an evaluation, a data-generation run, or a load test), because the cap is enforced on one accumulator holding the run’s whole spend. The increment that crosses the limit marks the run stopped and discards every queued item no worker has started; the workers find an empty queue and finish, and the ArenaRun fails with reason BudgetExceeded (a Warning event on the run, and budgetBreached, totalCost and drainedItems in its status.result.summary).

Caution

spec.loadTest.budgetLimit has been removed. A job that still sets it runs with no cap at all: the field is pruned from an unknown schema rather than rejected, so nothing warns you. Move the value to spec.budgetLimit.

The value must be a bare decimal, ^[0-9]+(\.[0-9]+)?$. A currency symbol, a sign or an exponent is rejected by the API server on apply:

The ArenaJob "x" is invalid: spec.budgetLimit: Invalid value: "$25.00":
spec.budgetLimit in body should match '^[0-9]+(\.[0-9]+)?$'

The admission webhook checks the same thing, but the pattern is what you can rely on: webhook.enabled defaults to false, so in a default install the webhook is not running and the schema is the only validation there is.

The cap is a ceiling rather than an exact stop. Cost reaches the run’s accumulator as each request completes (through the queue for a run that drains one, and charged directly by a load test with a duration), so the overshoot is the requests already in flight when the cap is crossed (at most concurrency requests’ worth). Every worker learns the cap is spent at its next completion and starts nothing further. The cap binds on the run’s total; it is not divided between workers.

loadTest

Settings specific to load testing jobs (used when type: loadtest).

Field	Type	Default	Description
concurrency	integer	1	Maximum number of work items in flight across all workers. The ceiling is divided evenly across the worker pods, rounded up, and each pod enforces its share without reading the others’ counts. Minimum 1.
vusPerWorker	integer	1	Number of virtual users (concurrent goroutines) per worker pod. Each VU independently pops, executes, and reports work items. Minimum 1.
duration	string	-	How long to apply load for (e.g. "10m"). Setting it changes where the work comes from — see below.
rate	integer	-	Target arrivals per second across all workers. Selects the open model; requires duration. Minimum 1.
ramp	object	-	Linear concurrency ramp-up / ramp-down (see below).
thresholds	[]object	-	SLO targets evaluated after the load test completes (see below).

loadTest.duration: a clock instead of a matrix

Without duration, a load test’s volume is scenarios × providers × trials, and its length is items × latency ÷ concurrency, which varies because latency is also what the test measures.

With duration, no matrix is queued. Each worker generates from the scenario × provider pool until the clock runs out, and trials is ignored. status.result.summary.timeSeries carries per-second rps, error rate and latency percentiles.

loadTest.rate: offered arrivals instead of a ceiling

rate switches the run from a closed model to an open one:

	driver	concurrency is
no rate	as fast as the ceiling allows	the target
rate set	an arrival schedule	a safety ceiling

Arrivals the workers cannot keep up with are reported, never queued. Gate on it with the arrival_lag threshold.

spec:
  type: loadtest
  loadTest:
    duration: 10m       # run for ten minutes, not for N items
    rate: 20            # offer twenty arrivals a second
    concurrency: 50     # now a safety limit, not the driver
    thresholds:
      - metric: arrival_lag
        operator: "<"
        value: "0.02"   # fail if 2% of arrivals were missed

loadTest.ramp

Controls how concurrency changes over the course of a load test. Both values are duration strings (e.g. "2m", "30s").

Field	Type	Description
up	string	Duration to linearly ramp from 0 to target concurrency at the start.
down	string	Duration to linearly ramp from target concurrency to 1 at the end. With loadTest.duration set, the ramp-down occupies the last down of the run. Without it, the ramp-down starts once a worker’s last completed item reports fewer than concurrency × 2 items still pending.

loadTest.thresholds

Each threshold is an SLO gate evaluated after the load test completes. The job fails if any threshold is violated.

Field	Type	Required	Description
metric	string	Yes	Metric to evaluate (see allowed values below).
operator	string	Yes	Comparison operator: <, >, <=, >=.
value	string	Yes	Target value to compare against.

Allowed metric values: latency_avg, latency_p50, latency_p90, latency_p95, latency_p99, ttft_avg, ttft_p50, ttft_p90, ttft_p95, ttft_p99, error_rate, pass_rate, total_cost, rate_limit_rate, arrival_lag.

Metric	Definition
latency_avg	Mean item latency, from the run’s counters. Available once one item has completed.
latency_p50, _p90, _p95, _p99	Percentile of the run’s latency distribution.
ttft_avg, ttft_p50, _p90, _p95, _p99	Mean or percentile of time-to-first-token, over the items that reported one.
error_rate	Failed items ÷ total items.
pass_rate	Passed items ÷ total items.
rate_limit_rate	Items whose error text matches a rate-limit phrase (for example “rate limit”, “too many requests”, “quota exceeded”, “resource exhausted”, “overloaded”) or contains a standalone 429, ÷ total items.
arrival_lag	Scheduled arrivals the generator could not keep up with ÷ arrivals offered.
total_cost	Accumulated cost, in budgetCurrency.

An item’s latency is the time the system under test spent answering it: the sum of its provider calls (for an agentRef, each call is one round trip to the agent, so the agent’s own tool calls are inside it). The worker’s own time (reading the run, building the engine, recording the session, running assertions) is not part of it, so a load test measures the target rather than itself. Nor are tools the worker executes itself (a scenario’s mock and HTTP tools) or the self-play user simulator’s provider calls: in a multi-turn self-play conversation, only the assistant turns are timed. When no provider call reports a duration, the engine’s execution time is used instead. The whole item, overhead included, is recorded separately on each item result as the itemDurationMs metric.

Latency percentiles are answered from the run’s latency distribution. A load test keeps one when it has a duration, or when it drains a matrix with concurrency or vusPerWorker above 1. An enumerated load test left at the defaults (concurrency: 1, vusPerWorker: 1) runs sequentially, keeps no distribution, and its percentile rules report unavailable.

arrival_lag only means anything for a run given a rate: a run with no arrival schedule reports unavailable rather than a zero, because a paced run that kept up also reports zero, and the two are different facts. A metric nobody could measure reports unavailable and passes.

ttft_* needs a provider that reports a first token, meaning a fleet entry (agentRef): the time is read off the agent’s stream after execution. It also needs a distribution, on the same terms as the latency percentiles above. A raw providerRef run measures total duration only. TTFT is kept as its own distribution rather than folded into latency’s, so a run that measured no first token reports unavailable instead of a wrong total-duration number.

value formats:

• Latency / TTFT metrics: a duration string (e.g. "3s", "500ms").
• Rate metrics (error_rate, pass_rate, rate_limit_rate, arrival_lag): a float string between 0 and 1 (e.g. "0.01", "0.95"). A target outside that range is rejected: error_rate < 50 would otherwise parse and pass every run.
• Cost metric (total_cost): a numeric string (e.g. "50.00").

The ArenaRun records each result in status.result.summary as threshold:<metric> (for example 3.20s < 3s FAIL; unavailable or parse error when the rule could not be judged) plus thresholds_passed (for example 3/4 passed). A violated threshold moves the run to Failed with reason TestsFailed. An enumerated load test also fails when any item fails; a run with a duration fails only on its thresholds.

When the admission webhook is running (webhook.enabled: true; it defaults to false), a value in none of these forms is rejected on apply, as are an unparseable duration or ramp value and a rate without a duration. Whether or not the webhook caught it, a value that cannot be read is reported as parse error at evaluation and fails the run rather than passing it.

spec:
  type: loadtest
  budgetLimit: "25.00"
  budgetCurrency: USD
  loadTest:
    concurrency: 50
    vusPerWorker: 10
    ramp:
      up: 2m
      down: 30s
    thresholds:
      - metric: latency_p95
        operator: "<"
        value: "3s"
      - metric: error_rate
        operator: "<="
        value: "0.01"

dataGen

Settings specific to data generation jobs (used when type: datagen).

Field	Type	Default	Description
count	integer	100	Number of items to generate
format	string	“jsonl”	Output format: jsonl or json

spec:
  type: datagen
  dataGen:
    count: 1000
    format: jsonl

csv is rejected at admission: a conversation is a nested list of turns, and flattening it into columns loses that structure.

Where the generated records go

Each generated conversation lands in two places, holding the same content:

• Sessions: recorded as an ordinary session, tagged kind:corpus and arena-run:<run name>. This is what the dashboard’s Corpus tab reads, and what you query: GET /api/v1/sessions?tag=kind:corpus&tag=arena-run:<run>. A datagen run records these without sessionRecording being set; see sessionRecording.
• A file, in format, written to the run’s output target: the export, downloaded or fed to training, written whether or not sessions were recorded.

There is no separate corpus store to configure.

workers

Configure the worker pool for job execution.

Field	Type	Default	Description
replicas	integer	1	Number of worker replicas
podOverrides	object	-	Customizes the worker Job Pods (scheduling, ServiceAccount, CSI secret-stores, custom envFrom for provider credentials, etc.). See podOverrides

spec:
  workers:
    replicas: 10

replicas sets the worker Job’s parallelism and completions: that many worker pods run each ArenaRun’s work concurrently.

A workspace’s spec.quotas.arena.maxWorkersPerJob caps it: a run asking for more runs at the cap, and its status.workers records the count it started with. With the licence webhooks enabled, an explicit replicas above the cap is refused on create, or when an update raises it, and the licence’s worker-replica limit is checked against the capped count, as the controller checks it.

providers

Maps group names to provider groups. Group names correspond to the arena config file’s provider groups — the group: value on each providers: entry in config.arena.yaml (e.g. "default", "judge").

When providers is set, provider YAML files from the arena bundle are ignored and the worker resolves providers directly from CRDs.

Each entry is an ArenaProviderEntry, with one of the following fields set:

Field	Type	Required	Description
providerRef	object	Conditional	Reference to a Provider CRD
providerRef.name	string	Yes	Name of the Provider resource
providerRef.namespace	string	No	If set, must equal the ArenaJob’s own namespace; a Provider in any other namespace is refused: the run’s JobCreated condition goes False with reason JobCreationFailed and the controller retries with backoff
agentRef	object	Conditional	Reference to an AgentRuntime CRD
agentRef.name	string	Yes	Name of the AgentRuntime resource

Caution

Each entry must set exactly one of providerRef or agentRef. The providers map is excluded from CRD schema validation, so the API server does not check it. The admission webhook rejects a bad entry when webhook.enabled is true. Otherwise the ArenaRun controller refuses it: the run moves to Failed with reason InvalidProviderEntry, and the message names the group and the entry’s index (array mode) or key (map mode).

Agents and LLM providers are interchangeable in the scenario × provider matrix. An agentRef entry causes the worker to connect to the agent over WebSocket instead of making direct LLM API calls.

Array mode vs map mode

Each provider group accepts two shapes, and the two can be mixed across groups within the same job:

• Array mode (default) — a list of entries. The group is a pool of test providers, and every entry is run against every selected scenario.
• Map mode — an object keyed by config-provider-ID. Each key is the exact provider ID the arena config expects, mapped 1:1 to a CRD entry. Use this when the arena config references specific provider IDs (for example, the deploy wizard emits map mode so the config’s provider IDs resolve exactly). Map-mode groups (judges, self-play roles) do not add entries to the scenario × provider matrix.

Array mode:

spec:
  providers:
    default:
      - providerRef:
          name: gpt4-prod
      - providerRef:
          name: claude-sonnet

Map mode:

spec:
  providers:
    default:
      # config-provider-ID → CRD
      primary:
        providerRef:
          name: gpt4-prod
      secondary:
        providerRef:
          name: claude-sonnet

Example: Separate Judge Provider

Use a dedicated provider group for the judge (evaluator) model. The group name (judge) must match a group: value used in the arena config’s providers: list:

spec:
  providers:
    default:
      - providerRef:
          name: gpt4-prod
      - providerRef:
          name: claude-sonnet
    judge:
      - providerRef:
          name: claude-opus

Example: Agent Entry

Reference a deployed AgentRuntime instead of a raw LLM provider. The worker connects to the agent’s WebSocket endpoint:

spec:
  providers:
    default:
      - agentRef:
          name: my-support-agent

Example: Self-Play with Mixed Types

Mix LLM providers and agents in a self-play evaluation:

spec:
  providers:
    selfplay:
      - providerRef:
          name: gpt4-prod
      - agentRef:
          name: my-agent-v2
    judge:
      - providerRef:
          name: claude-opus

toolRegistries

List of ToolRegistry CRD references, in the job’s namespace, whose discovered tools override how the arena config’s tools are executed. The tool YAML files in the bundle still define the tools; a tool whose name matches a discovered tool is rewritten to call that tool’s endpoint over HTTP.

Field	Type	Required	Description
name	string	Yes	Name of the ToolRegistry resource

spec:
  toolRegistries:
    - name: production-tools

How Tool Registries Work

1. The worker reads each referenced ToolRegistry CRD
2. It collects the tools listed in each registry’s status.discoveredTools
3. For every tool file in the arena config whose tool name matches a discovered tool, the worker switches the tool to an HTTP executor at the discovered endpoint (method POST unless the file sets one) and takes the registry’s description
4. Tools with no matching discovered tool, and mcp_servers, are used as written

Example: Multiple Tool Registries

spec:
  toolRegistries:
    - name: core-tools
    - name: billing-tools

Combining Providers and Tool Registries

Example combining providers and toolRegistries:

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: production-eval
spec:
  sourceRef:
    name: my-source
  arenaFile: config.arena.yaml
  providers:
    default:
      - providerRef:
          name: gpt4-prod
      - providerRef:
          name: claude-sonnet
    judge:
      - providerRef:
          name: claude-opus
  toolRegistries:
    - name: production-tools
  workers:
    replicas: 5
  output:
    type: s3
    s3:
      bucket: arena-results
      prefix: "evals/"

output

Configure where job results are stored.

Field	Type	Required	Description
type	string	Yes	Destination type: s3, pvc
s3	object	Conditional	S3 configuration (when type is s3)
pvc	object	Conditional	PVC configuration (when type is pvc)

S3 Output

Field	Type	Required	Description
bucket	string	Yes	S3 bucket name
prefix	string	No	Key prefix for objects
region	string	No	AWS region
endpoint	string	No	Custom S3-compatible endpoint
secretRef	object	No	Credentials secret reference (keys: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

spec:
  output:
    type: s3
    s3:
      bucket: arena-results
      prefix: "evals/nightly/"
      region: us-west-2
      secretRef:
        name: s3-credentials

PVC Output

Field	Type	Required	Description
claimName	string	Yes	PVC name
subPath	string	No	Subdirectory within PVC

spec:
  output:
    type: pvc
    pvc:
      claimName: arena-results-pvc
      subPath: "evals/"

Without output

A run whose job sets no output keeps its engine output on the workspace content volume, in its own directory arena/runs/{run}/. The output holds the per-item JSON results, so you have them whether or not the run records sessions to session-api.

Behaviour	Detail
When	The run’s content comes from the workspace content volume, which is the case for every project run when the Arena controller has --workspace-content-path set. Otherwise the worker writes to its scratch space, and the output is gone when the pod exits
Worker mount	The content volume, read-write, at /arena-output, with a subPath of the run’s directory alone. The project content stays a separate read-only mount
Run marker	The annotation arena.omnia.altairalabs.ai/output: arena/runs/{run} and the finalizer arena.omnia.altairalabs.ai/output
Download	GET /api/v1/workspaces/{name}/arena/runs/{runName}/output returns the directory as a zip, for a workspace viewer. The run page offers it as Download output once the run has finished or been cancelled
Removal	Deleting the run removes the directory. This includes runs pruned by successfulRunsHistoryLimit and failedRunsHistoryLimit
Failure	If the Arena controller cannot create the directory or mark the run, it logs the error and the worker writes to its scratch space. The run proceeds either way
No content volume	An Arena controller without --workspace-content-path adds no finalizer. One that has the finalizer on a run but no volume mounted logs and releases the run, leaving the directory

The Arena controller removes the finalizer. If the controller is gone, for example after Enterprise is disabled or the Arena controller is uninstalled, a deleted run waits with a deletion timestamp. Remove the finalizer by hand, then delete the run’s directory from the volume yourself if you want it gone:

kubectl patch arenarun <run> -n <namespace> --context <context> --type=json \
  -p '[{"op":"remove","path":"/metadata/finalizers/<index>"}]'

<index> is the position of arena.omnia.altairalabs.ai/output in the run’s metadata.finalizers, counted from 0:

kubectl get arenarun <run> -n <namespace> --context <context> \
  -o jsonpath='{.metadata.finalizers}'

When it is the run’s only finalizer, a merge patch clears it without the index:

kubectl patch arenarun <run> -n <namespace> --context <context> --type=merge \
  -p '{"metadata":{"finalizers":null}}'

schedule

Configure scheduled/recurring execution. Each due tick creates a new ArenaRun (trigger Schedule). Without a schedule (or with an empty cron), the job creates exactly one run (trigger Manual) when it is first reconciled.

Field	Type	Default	Description
cron	string	-	Cron expression for scheduling
timezone	string	“UTC”	Timezone for cron
concurrencyPolicy	string	“Forbid”	Allow starts the new run alongside any still going; Forbid skips the tick (recording it as passed, with a RunSkipped event) while a run is active; Replace cancels the active runs (sets their spec.cancelled) and then starts the new one

spec:
  schedule:
    cron: "0 2 * * *"  # 2am daily
    timezone: "America/New_York"
    concurrencyPolicy: Forbid

Under a workspace spec.quotas.arena.maxJobsPerDay, a tick is also skipped, recorded as passed with a Warning RunSkipped event, when the runs already started or waiting in the last 24 hours use up the allowance. A run created by a tick that the workspace’s maxConcurrentJobs cannot admit yet waits Pending instead; under Forbid that waiting run is active, so later ticks are skipped until it finishes.

suspend

Stops the job starting new runs, without deleting it. Runs already going are left alone — to stop one, set spec.cancelled on the ArenaRun. While suspended, status.nextScheduleTime is cleared and a Suspended event is recorded. Default false.

spec:
  suspend: true

successfulRunsHistoryLimit / failedRunsHistoryLimit

How many finished ArenaRuns the job keeps. Older ones beyond the limit are deleted. Defaults: 3 successful, 1 failed (a Cancelled run counts as failed). Minimum 0. The per-scenario detail of every run lives in session-api, so pruning a run object does not lose its results.

spec:
  successfulRunsHistoryLimit: 7
  failedRunsHistoryLimit: 3

serviceGroup

The Workspace service group whose session-api this job’s runs record into, and whose redis, when it sets one, holds the runs’ work queue (see the ArenaRun’s status.queue). For the session-api, it is resolved only when sessionRecording is true, or the job is a datagen job. For the queue, it is resolved whenever any group in the workspace sets a redis; a name the workspace does not declare then fails the run with ServiceGroupNotFound. When omitted, a workspace with exactly one service group resolves to it. A workspace with several is ambiguous: the controller does not pick one, and the run moves to Failed with reason ServiceGroupAmbiguous. A group the workspace does not have, a workspace with no service groups in its status, or a group that declares no session-api, leaves the worker Job uncreated: the run’s JobCreated condition goes False with reason JobCreationFailed and the controller retries with backoff. Lowercase DNS label, at most 63 characters.

spec:
  serviceGroup: default

verbose

Enables verbose/debug logging for arena execution. When enabled, the worker pods run with LOG_LEVEL=debug and ARENA_VERBOSE=true, which turns on the arena engine’s and PromptKit SDK’s verbose logging.

spec:
  verbose: true

sessionRecording

Enables writing session data to session-api during execution. When false (default), no sessions are created and no events are recorded, reducing session-api load during high-volume load tests. Telemetry and traces are unaffected.

A datagen run records sessions whether or not this is set, because its sessions are its corpus. If the workspace has no session-api to record into and the field is not set, the run still writes its export, and emits a CorpusNotRecorded warning event; with the field set to true, the same problem stops the run, as for any other type.

When it is true, a load-test conversation against an agentRef entry is recorded by the agent’s facade as its own session, and the worker adds arena context and any self-play or judge calls to that session. A conversation against a providerRef entry is recorded by the worker’s own session manager.

spec:
  sessionRecording: true

ttlSecondsAfterFinished

How long Kubernetes keeps each run’s finished worker Job (and its pods) before deleting it. Sets the worker Job’s ttlSecondsAfterFinished; defaults to 3600 (one hour) when unset. It does not delete ArenaJobs or ArenaRuns — run history is bounded by the history limits above.

spec:
  ttlSecondsAfterFinished: 86400  # 24 hours

Status Fields

An ArenaJob’s status describes scheduling. Per-execution state is on each ArenaRun.

Field	Description
active	References to the ArenaRuns that have not finished
lastRunRef	Reference to the most recently created ArenaRun
lastScheduleTime	The last cron tick that was acted on (a run started or skipped)
nextScheduleTime	The next cron tick; cleared while suspend is true
observedGeneration	Most recent generation observed by the controller

conditions

Type	Description
Ready	True (reason Scheduled) while the job can schedule runs; False with reason InvalidSchedule when the cron expression cannot be parsed
Stalled	True when the job cannot make progress without a spec change (an invalid schedule); False otherwise

ArenaRun

An ArenaRun (short name arun) is one execution of an ArenaJob. The job’s controller creates it; you do not normally write one yourself.

Spec field	Description
jobRef.name	The ArenaJob this run was created from
trigger	Schedule (a cron tick) or Manual (default)
template	The ArenaJob spec frozen when the run was created. Every field documented above is read from here
cancelled	Set to true to stop the run: the controller deletes the worker Job, drains the run’s remaining queued work, and moves the run to Cancelled. Has no effect once the run is Succeeded, Failed or Cancelled

kubectl -n arena patch arenarun nightly-eval-x7k2p \
  --type=merge -p '{"spec":{"cancelled":true}}'

phase

Value	Description
Pending	Waiting to start: the ArenaProject is not Ready yet, or the workspace’s spec.quotas.arena has not admitted the run (WithinWorkspaceQuota=False)
Running	The worker Job has been created and is executing
Succeeded	Completed successfully
Failed	Failed, including a ref that names nothing in the project (reason RefNotFound), a commit that cannot be exported (SnapshotUnexportable), a provider entry that sets both or neither of providerRef and agentRef (InvalidProviderEntry), an ambiguous service group (ServiceGroupAmbiguous), a named service group the workspace does not declare when its groups set their own Redis (ServiceGroupNotFound), a budget stop, a failed threshold, a licence violation or a work matrix over the item cap
Cancelled	Stopped by spec.cancelled

source

The commit the run executes, set once when the run resolves template.ref and never changed.

Field	Description
repository	The project’s repository: its URL for an external project, hosted for a hosted one
ref	The ref the run asked for; the project’s default branch when the template named none. A run started from the project editor (or the operator’s start-run route) pins spec.ref to a SHA and shows the ref the person asked for here, from the job’s arena.omnia.altairalabs.ai/requested-ref annotation
commit	The commit SHA ref resolved to
contentPath	The commit’s snapshot, relative to the workspace content subtree (arena/sources/<project>/.arena/versions/<commit>), which the workers mount

queue

The Redis the run’s work queue lives on, set once before the worker Job is created and never changed. See services[].redis for how it is resolved.

Field	Description
source	ServiceGroup when the run’s service group sets a redis; Default for the operator-wide arena queue (enterprise.arena.queue.redis)
serviceGroup	The service group the run resolved to: template.serviceGroup, or the workspace’s only group. Empty when no group applies
secretRef	name and key of the Secret, in the run’s namespace, holding the queue’s Redis URL. Set for ServiceGroup only. It is the group’s own existingSecret, or <run>-queue-redis (key url), which the controller writes for a url, host or serviceRef group and which is owned by the run
configHash	A fingerprint of the group’s redis when the run started. If the group’s redis later differs, the run gets the QueueRedisDrifted condition and stays on its queue

A run that was already started when this field arrived is pinned to Default.

progress

Field	Description
total	Total number of work items
completed	Successfully completed items
failed	Failed items
pending	Pending items

result

The run’s headline result. The per-scenario, per-provider detail is streamed to session-api as the run executes.

Field	Description
summary	Aggregated result metrics (string map)

An evaluation run’s summary holds these keys. Every value is a string.

Key	Description
passRate	Percentage of items that passed, to one decimal place ("100.0")
totalItems	Items executed
passedItems	Items that passed
failedItems	Items that failed
avgDurationMs	Average item duration in milliseconds
totalTokens	Tokens used; set only when above zero
totalCost	Cost to four decimal places; set only when above zero
details	JSON string with the breakdown: scenarios, providers, cells (one per scenario and provider pair), assertions and errors

conditions

Type	Description
Ready	Overall readiness of the run
SourceValid	Referenced ArenaProject exists and is Ready, and the run’s commit is pinned in status.source
JobCreated	Worker Kubernetes Job has been created
Progressing	Run is executing workers
QueueRedisDrifted	True (reason ServiceGroupRedisChanged) when the run’s service group redis has changed since the run was pinned. The run keeps its queue. Set back to False (reason Pinned) if the group returns to the pinned Redis
WithinWorkspaceQuota	Present while the workspace sets spec.quotas.arena, written before the run starts. True (reason QuotaSatisfied) when the run is admitted. False while it waits, Pending with Ready=False (reason WaitingForWorkspaceQuota): reason WaitingForConcurrentRunSlot when maxConcurrentJobs runs are running, WaitingForDailyRunQuota when maxJobsPerDay runs were admitted in the last 24 hours (the message gives when the next slot opens), WaitingForQuotaAdmission until the quota has counted the run, AdmissionReleased when the run was admitted but did not start its workers within 15 minutes and gave its slot back. Once admitted, a Ready=False WaitingForWorkspaceQuota is removed. False with reason WorkersClamped when the run is admitted but asks for more workers than maxWorkersPerJob: it runs at the cap. A waiting run is never failed, and is admitted oldest first
OutputPersisted	Set once the run finishes, only for S3 output. True (reason OutputUploaded) when every worker uploaded; False (reason OutputUploadFailed, with a matching Warning event) when any worker’s upload failed. It does not change the run’s phase.

Other fields

Field	Description
startTime	When the run started
completionTime	When the run completed
activeWorkers	Current number of active worker pods
workers	The worker replicas the worker Job was created with: template.workers.replicas, capped at the workspace’s maxWorkersPerJob. Set once, so a quota changed later never resizes a running run. A run whose Job predates the field gets the Job’s parallelism
observedGeneration	Most recent generation observed by the controller

Complete Examples

Basic Evaluation Job

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: basic-eval
  namespace: arena
spec:
  sourceRef:
    name: my-source
  arenaFile: config.arena.yaml

Multi-Worker Evaluation

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: parallel-eval
  namespace: arena
spec:
  sourceRef:
    name: provider-comparison
  type: evaluation
  workers:
    replicas: 10
  output:
    type: s3
    s3:
      bucket: arena-results
      prefix: "evals/parallel/"

Scheduled Nightly Evaluation

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: nightly-eval
  namespace: arena
spec:
  sourceRef:
    name: production-tests
  type: evaluation
  workers:
    replicas: 5
  output:
    type: s3
    s3:
      bucket: arena-results
      prefix: "evals/nightly/"
  schedule:
    cron: "0 2 * * *"
    timezone: "UTC"
  ttlSecondsAfterFinished: 604800  # 7 days

Load Testing Job

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: provider-loadtest
  namespace: arena
spec:
  sourceRef:
    name: load-test-source
  type: loadtest
  trials: 5000
  sessionRecording: false
  loadTest:
    concurrency: 100
    vusPerWorker: 20
    ramp:
      up: 2m
      down: 1m
    thresholds:
      - metric: latency_p95
        operator: "<"
        value: "3s"
      - metric: error_rate
        operator: "<="
        value: "0.02"
  workers:
    replicas: 5
  output:
    type: s3
    s3:
      bucket: loadtest-results
      prefix: "loadtests/"

Data Generation Job

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: synthetic-data
  namespace: arena
spec:
  sourceRef:
    name: datagen-source
  type: datagen
  dataGen:
    count: 10000
    format: jsonl
  workers:
    replicas: 4
  output:
    type: pvc
    pvc:
      claimName: generated-data
      subPath: "batch-001/"

Workflow

1. Create an ArenaProject — a bundle containing the arena config file (config.arena.yaml) plus scenarios and prompts.
2. Create an ArenaJob — reference the project, select the config file with arenaFile, and specify execution parameters (job type, providers, workers).
3. Monitor progress — the job creates an ArenaRun per execution (kubectl get arun); watch the run’s status.progress for completion.
4. Retrieve results — access results from the configured output destination.

Related Resources

• ArenaProject: Defines the bundle source (git, OCI, S3, ConfigMap, or workspace)
• Arena Config File: Schema of the config.arena.yaml file inside the bundle
• Provider: LLM provider configuration