Arena Config File

Enterprise Feature

The arena config file drives the enterprise Arena Fleet feature. The Arena CRDs come from the omnia-crds chart, which installs them by default; the Arena controllers run only when enterprise.enabled=true in the app chart’s values. See Installing a License for details.

The arena config file (conventionally config.arena.yaml) is a YAML file that lives inside an ArenaProject bundle. It ties together the prompts, scenarios, providers, tools, and evaluation settings that a run uses. An ArenaJob selects it with spec.arenaFile (default config.arena.yaml; one file, glob patterns are rejected), and the arena worker loads it from the bundle when each of the job’s ArenaRuns executes.

This is a file, not a CRD

There is no ArenaConfig Kubernetes resource. Do not kubectl apply this document — it is a plain YAML file committed alongside your scenarios in the ArenaProject bundle. The Arena CRDs are ArenaProject, ArenaJob, ArenaRun (created by an ArenaJob, one per execution), and ArenaTemplateSource.

File shape

The file uses a lightweight PromptKit envelope. Everything below lives under spec:.

apiVersion: promptkit.altairalabs.ai/v1alpha1
kind: Arena
metadata:
  name: customer-support-eval
spec:
  # ... fields documented below

Most sections reference sibling files inside the bundle, relative to the config file’s directory, rather than inlining content.

Spec Fields

prompt_configs

Prompt configurations available to scenarios, referenced by file.

Field	Type	Required	Description
id	string	Yes	Prompt identifier referenced by scenarios
file	string	Yes	Path to the prompt YAML within the bundle

spec:
  prompt_configs:
    - id: assistant
      file: prompts/assistant.yaml

providers

The provider entries used by scenarios. Each entry references a provider YAML file and, optionally, assigns the provider to a group.

Field	Type	Required	Description
file	string	Yes	Path to the provider YAML within the bundle
group	string	No	Provider group name (e.g. default, judge)

spec:
  providers:
    - file: providers/gpt4.provider.yaml
      group: default
    - file: providers/claude-opus.provider.yaml
      group: judge

Groups connect the file to the ArenaJob

The group value is the join key with ArenaJob.spec.providers. When an ArenaJob sets spec.providers, the provider YAML files here are ignored and the worker resolves providers from Provider/AgentRuntime CRDs instead — matching each ArenaJob provider-group name to the group used here. Leave providers empty (providers: []) when you intend to supply all providers from CRDs.

scenarios

Scenario files to run, each referenced by file.

Field	Type	Required	Description
file	string	Yes	Path to the scenario YAML within the bundle

spec:
  scenarios:
    - file: scenarios/billing.scenario.yaml
    - file: scenarios/support.scenario.yaml

judges and judge_defaults

Named judge (evaluator) targets. Each judge inherits its model from the referenced provider ID.

Field	Type	Required	Description
name	string	Yes	Judge identifier used in assertions
provider	string	Yes	Provider ID reference (must exist in spec.providers)

spec:
  judges:
    - name: quality-judge
      provider: claude-opus

evals and tools

Additional evaluation definitions and tool definitions, each referenced by file.

spec:
  evals:
    - file: evals/quality.eval.yaml
  tools:
    - file: tools/search.tool.yaml

Note

When an ArenaJob sets spec.toolRegistries, the tool files listed here still define the tools. For each tool whose name matches a tool discovered by one of those ToolRegistries, the worker rewrites the tool to an HTTP executor pointing at the registry’s endpoint (and takes the registry’s description). Tools with no match, and mcp_servers, are left as written.

mcp_servers

Model Context Protocol (MCP) servers to expose to the model during a run. An entry sets command for stdio, url for HTTP, or source for a host-provisioned endpoint. Nothing enforces that only one of these is set. If both command and url are present, the server resolves the transport by precedence instead of rejecting the entry: an explicit transport value first, then url (as sse), then command (as stdio).

Field	Type	Description
name	string	Server identifier
command / args / env	string / []string / map	stdio transport (local subprocess)
url / headers	string / map	HTTP transport
transport	string	stdio, sse, or streamable_http
tool_filter	object	allowlist / blocklist of tool names

spec:
  mcp_servers:
    - name: filesystem
      command: mcp-server-filesystem
      args: ["--root", "/data"]

self_play

Configure self-play (agent-vs-simulated-user) evaluation. Self-play is enabled whenever this section is present.

Field	Type	Description
personas	[]object	Persona files, each { file: <path> }
roles	[]object	Role-to-provider mapping, each { id: <role>, provider: <provider-id> }

spec:
  self_play:
    personas:
      - file: personas/frustrated-customer.yaml
    roles:
      - id: user
        provider: gpt4

defaults

Default execution settings applied to every scenario unless it overrides them.

Field	Type	Default	Description
temperature	number	-	Sampling temperature
max_tokens	integer	-	Maximum output tokens
seed	integer	-	Deterministic sampling seed
concurrency	integer	-	Parallel runs
run_timeout	string	“5m”	Per-run timeout (e.g. "30s")
fail_on	[]string	-	Conditions that mark the run failed
output	object	-	Output directory and formats

spec:
  defaults:
    temperature: 0.5
    max_tokens: 500
    seed: 42
    run_timeout: 2m
    output:
      dir: out
      formats:
        - json

globals

Cross-cutting settings applied additively to every scenario.

Field	Type	Description
conversation_assertions	[]object	Assertions appended to every scenario’s own conversation assertions

Complete Example

A minimal bundle layout:

my-bundle/
├── config.arena.yaml
├── prompts/
│   └── assistant.yaml
├── providers/
│   └── gpt4.provider.yaml
└── scenarios/
    ├── billing.scenario.yaml
    └── support.scenario.yaml

config.arena.yaml:

apiVersion: promptkit.altairalabs.ai/v1alpha1
kind: Arena
metadata:
  name: customer-support-eval
spec:
  prompt_configs:
    - id: assistant
      file: prompts/assistant.yaml

  providers:
    - file: providers/gpt4.provider.yaml
      group: default

  scenarios:
    - file: scenarios/billing.scenario.yaml
    - file: scenarios/support.scenario.yaml

  defaults:
    temperature: 0.5
    max_tokens: 500
    run_timeout: 2m
    output:
      dir: out
      formats:
        - json

Providers from CRDs

Leave the providers list empty when the ArenaJob supplies providers from Provider/AgentRuntime CRDs instead:

spec:
  # Providers supplied by ArenaJob.spec.providers (resolved from CRDs)
  providers: []

  scenarios:
    - file: scenarios/billing.scenario.yaml

Using the config file with an ArenaJob

An ArenaJob references the bundle via sourceRef and selects this file via arenaFile:

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: eval-001
  namespace: arena
spec:
  sourceRef:
    name: customer-support-source
  arenaFile: config.arena.yaml
  providers:
    default:
      - providerRef:
          name: gpt4-prod
    judge:
      - providerRef:
          name: claude-opus

Workflow

See Arena Fleet: Run Your First Evaluation for the workflow steps.

The config file feeds a chain of resources from the bundle to the run’s results:

flowchart LR
    config[config.arena.yaml] --> project[ArenaProject]
    project --> job[ArenaJob]
    job --> run[ArenaRun]
    run --> workers[Workers]
    workers --> results[Results]

Related Resources

• ArenaProject — the bundle source that contains this config file.
• ArenaJob — executes a run using this config file (spec.arenaFile).
• Provider — LLM provider configuration.
• ToolRegistry — tool definitions.