Set up Arena scheduled jobs

Enterprise

This guide shows how to set up scheduled Arena Fleet evaluations that run automatically on a recurring basis, such as nightly regression tests or weekly performance benchmarks.

Basic scheduled job

Add a schedule section to your ArenaJob to enable recurring execution:

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: nightly-regression
  namespace: default
spec:
  sourceRef:
    name: regression-config
  type: evaluation
  workers:
    replicas: 2
  schedule:
    cron: "0 2 * * *"  # Run at 2:00 AM daily
    timezone: "UTC"

Cron expression reference

The cron field uses standard cron syntax:

The five space-separated fields, in order, are:

Position	Field	Range
1	minute	0–59
2	hour	0–23
3	day of month	1–31
4	month	1–12
5	day of week	0–6 (Sunday = 0)

* * * * *
minute hour day-of-month month day-of-week

Common schedules

Schedule	Cron Expression	Description
Nightly	`0 2 * * *`	Every day at 2:00 AM
Hourly	`0 * * * *`	Every hour on the hour
Every 6 hours	`0 /6 * *`	At 0:00, 6:00, 12:00, 18:00
Weekly	`0 3 * * 0`	Sunday at 3:00 AM
Weekdays	`0 8 * * 1-5`	Monday-Friday at 8:00 AM
Monthly	`0 4 1 * *`	1st of each month at 4:00 AM

Timezone configuration

By default, schedules use UTC. Specify a timezone for local time scheduling:

spec:
  schedule:
    cron: "0 2 * * *"
    timezone: "America/New_York"  # 2:00 AM Eastern Time

Common timezones:

UTC (default)
America/New_York
America/Los_Angeles
Europe/London
Asia/Tokyo

Concurrency policy

Control what happens when a scheduled run would start while a previous run is still active:

spec:
  schedule:
    cron: "0 * * * *"
    concurrencyPolicy: Forbid  # Default

Policy	Behavior
`Forbid`	Skip the new run if previous is still active (default)
`Allow`	Run concurrently with previous run
`Replace`	Cancel the previous run and start new one

When to use each policy

Forbid (recommended for most cases):

Prevents resource contention
Ensures results don’t overlap
Safe default for evaluations

Allow:

Use when runs are independent
When you need guaranteed execution of every scheduled run
Ensure sufficient cluster resources

Replace:

When only the latest results matter
For long-running jobs that may need to be superseded
Use with caution as it cancels in-progress work

Complete example: nightly regression suite

apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: nightly-regression
  namespace: arena
  labels:
    team: platform
    environment: staging
spec:
  sourceRef:
    name: regression-suite
  type: evaluation

  evaluation:
    outputFormats:
      - json
      - junit

  workers:
    replicas: 5

  output:
    type: s3
    s3:
      bucket: arena-results
      prefix: "regression/nightly/"
      region: us-west-2
      secretRef:
        name: s3-credentials

  schedule:
    cron: "0 2 * * *"      # 2:00 AM daily
    timezone: "UTC"
    concurrencyPolicy: Forbid

  ttlSecondsAfterFinished: 604800  # Keep results for 7 days

Monitoring scheduled jobs

View schedule status

kubectl get arenajob nightly-regression -o yaml

The status shows scheduling information:

status:
  phase: Succeeded
  lastScheduleTime: "2025-01-18T02:00:00Z"
  nextScheduleTime: "2025-01-19T02:00:00Z"

List recent runs

Scheduled jobs create child jobs for each execution. View recent runs:

kubectl get arenajobs -l arena.omnia.altairalabs.ai/parent=nightly-regression

Check for missed runs

If a scheduled run was missed (e.g., controller was down), it will be noted in conditions:

kubectl describe arenajob nightly-regression | grep -A 5 Conditions

Suspending scheduled jobs

Temporarily pause a scheduled job without deleting it:

kubectl patch arenajob nightly-regression --type=merge -p '{"spec":{"suspend":true}}'

Resume the schedule:

kubectl patch arenajob nightly-regression --type=merge -p '{"spec":{"suspend":false}}'

Or in the manifest:

spec:
  suspend: true  # Pauses scheduling
  schedule:
    cron: "0 2 * * *"

Multiple schedules

To run the same evaluation on different schedules (e.g., quick checks hourly, full suite nightly), create separate ArenaJobs:

# Quick hourly check
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: quick-check-hourly
spec:
  sourceRef:
    name: quick-check-config  # Subset of scenarios
  schedule:
    cron: "0 * * * *"
  workers:
    replicas: 1
---
# Full nightly suite
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  name: full-suite-nightly
spec:
  sourceRef:
    name: full-suite-config  # All scenarios
  schedule:
    cron: "0 2 * * *"
  workers:
    replicas: 10

Integration with CI/CD

Triggering manual runs

Force an immediate run of a scheduled job:

# Create a one-time job from the same config
kubectl create -f - <<EOF
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: ArenaJob
metadata:
  generateName: manual-regression-
  namespace: arena
spec:
  sourceRef:
    name: regression-suite
  type: evaluation
  workers:
    replicas: 5
EOF

Alerting on failures

Configure alerts for scheduled job failures using Prometheus alerts:

groups:
  - name: arena-alerts
    rules:
      - alert: ArenaScheduledJobFailed
        expr: |
          arena_job_status{scheduled="true", phase="Failed"} == 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Scheduled Arena job {{ $labels.job_name }} failed"

Cleanup and retention

Automatic cleanup with TTL

Set ttlSecondsAfterFinished to automatically delete completed jobs:

spec:
  ttlSecondsAfterFinished: 604800  # 7 days
  schedule:
    cron: "0 2 * * *"

Manual cleanup

Delete old job runs:

# Delete jobs older than 7 days
kubectl delete arenajobs -l arena.omnia.altairalabs.ai/parent=nightly-regression \
  --field-selector 'status.completionTime<2025-01-11T00:00:00Z'

ArenaJob Reference: Complete schedule configuration options
Monitor Arena Jobs: Track scheduled job execution
Configure S3 Storage: Store results from scheduled runs