Skip to content

Install on OpenShift

Install Omnia on Red Hat OpenShift 4.11 or later so that every pod it runs is admitted by the default restricted-v2 SecurityContextConstraints.

  • OpenShift 4.11 or later, and oc or kubectl logged in as a cluster administrator
  • Helm v3
  • A StorageClass that provides ReadWriteMany volumes, such as the ocs-storagecluster-cephfs class of OpenShift Data Foundation, or an NFS provisioner your cluster administrator runs
  • A PostgreSQL and a Redis provisioned outside the chart

restricted-v2 assigns every namespace a UID range and admits a pod only with a runAsUser inside it. By default every Omnia pod pins runAsUser, runAsGroup and fsGroup to 65532, so restricted-v2 refuses it.

With openshift.enabled: true:

PodsChange
Chart pods: operator, arena controller, dashboard, management-api, doctor, PromptKit LSP, the compaction CronJob, helm test podsNo runAsUser, runAsGroup or fsGroup, including any you set in podSecurityContext or dashboard.podSecurityContext
Pods the operator builds: agents (facade, runtime, policy broker), sandboxes, sandbox egress proxies, browser enrolment podsThe operator runs with --omit-pod-ids and leaves the IDs unset
Arena worker JobsThe arena controller runs with --omit-pod-ids

OpenShift then assigns each pod a UID and fsGroup from its namespace’s range. Everything else restricted-v2 requires holds on every install: runAsNonRoot, allowPrivilegeEscalation: false, all capabilities dropped, the RuntimeDefault seccomp profile, and no host namespaces, ports or paths. session-api, memory-api, privacy-api and the eval worker never pin IDs.

The chart refuses to render with nfs.csiDriver.enabled, nfs.external.server or dev.enabled. The NFS CSI driver needs privileged node pods and hostPath volumes; an external NFS server without it reaches arena workers as an inline nfs volume, which restricted-v2 refuses; and the development subchart runs a privileged NFS server and fixed-UID Redis and Postgres pods.

  1. Install the CRD chart as in Install with Helm.

  2. Write a values file:

    openshift:
    enabled: true
    workspaces:
    storage:
    storageClass: ocs-storagecluster-cephfs
    accessModes: [ReadWriteMany]
    workspaceContent:
    persistence:
    storageClass: ocs-storagecluster-cephfs
    dashboard:
    ingress:
    enabled: true
    className: openshift-default
    host: omnia.apps.example.com
    annotations:
    route.openshift.io/termination: edge
    haproxy.router.openshift.io/timeout: 1h
  3. Install the app chart:

    Terminal window
    helm install omnia oci://ghcr.io/altairalabs/charts/omnia \
    -n omnia-system --create-namespace -f openshift-values.yaml
  4. Give workspace namespaces the release namespace’s UID and group ranges. Pods in different namespaces otherwise run as different UIDs and groups. The operator and arena controller write workspace content from the release namespace, and sandboxes and arena workers read and write it from each workspace namespace, so they must share a group. Read the release namespace’s ranges:

    Terminal window
    kubectl get namespace omnia-system -o jsonpath='{.metadata.annotations}'

    Set the same openshift.io/sa.scc.uid-range and openshift.io/sa.scc.supplemental-groups on each Workspace’s namespace when it is created, through spec.namespace.annotations:

    apiVersion: omnia.altairalabs.ai/v1alpha1
    kind: Workspace
    metadata:
    name: team-a
    spec:
    namespace:
    name: team-a
    create: true
    annotations:
    openshift.io/sa.scc.uid-range: 1000680000/10000
    openshift.io/sa.scc.supplemental-groups: 1000680000/10000

    OpenShift allocates ranges only to a namespace that has none, so these annotations must be on the namespace before its first pod. For a namespace that already exists, set them with kubectl annotate --overwrite and restart its pods.

OpenShift turns an Ingress into a Route. Set dashboard.ingress.className to openshift-default, or to the IngressClass of another IngressController. OpenShift creates one Route per path the chart’s Ingress declares, so /api reaches management-api and everything else reaches the dashboard. The generated Routes carry the Ingress’s annotations.

Agent consoles in the dashboard are WebSockets, and the router closes a connection idle for 30 seconds by default. Add haproxy.router.openshift.io/timeout: 1h to dashboard.ingress.annotations. Set dashboard.baseUrl when the public URL differs from https://<dashboard.ingress.host>.

Pick one.

  • Gateway API. OpenShift 4.19 and later ship a Gateway API controller in the Ingress Operator. Create a GatewayClass with controllerName: openshift.io/gateway-controller/v1, set gateway.className to its name, and follow Expose agents. The operator lists the endpoints of HTTPRoutes on the agent’s Connectivity tab.

  • A Route per agent. The backend is the agent’s Service, which has the agent’s name, on its facade port:

    apiVersion: route.openshift.io/v1
    kind: Route
    metadata:
    name: my-agent
    namespace: team-a
    annotations:
    haproxy.router.openshift.io/timeout: 1h
    spec:
    host: my-agent.apps.example.com
    to:
    kind: Service
    name: my-agent
    port:
    targetPort: facade
    tls:
    termination: edge

    Set haproxy.router.openshift.io/timeout above the longest pause in a WebSocket conversation. The operator discovers HTTPRoutes only, so a Route endpoint does not appear on the Connectivity tab.

Omnia references its images by tag under ghcr.io/altairalabs. In a disconnected cluster, mirror that prefix and add an ImageTagMirrorSet so that every node pulls from the mirror without any chart change:

apiVersion: config.openshift.io/v1
kind: ImageTagMirrorSet
metadata:
name: omnia
spec:
imageTagMirrors:
- source: ghcr.io/altairalabs
mirrors:
- registry.example.com/altairalabs

An ImageDigestMirrorSet does not apply, because the images are not pulled by digest. Alternatively, point each image at the mirror with the chart’s image registry override or its per-image repository values. The sandbox classes name their own images in sandbox.image, sandbox.promptarenaClass.image and sandbox.browserClass.image.

Check that every Omnia pod runs under restricted-v2:

Terminal window
kubectl get pods -n omnia-system \
-o custom-columns='NAME:.metadata.name,SCC:.metadata.annotations.openshift\.io/scc'

Run the same command against a workspace namespace once an agent is deployed there. Every row shows restricted-v2.

  • Sandbox images must run as an arbitrary UID with GID 0. The shipped sandbox classes put HOME and every other writable path on scratch volumes, and their images already run as 65532 rather than their build user. A custom sandbox class image that writes into a directory owned by its build user fails.
  • The Prometheus, Grafana, Loki, Tempo, Alloy and KEDA subcharts are third-party charts. Use OpenShift’s monitoring stack and the Custom Metrics Autoscaler operator instead.
  • Omnia ships no OLM bundle. Install it with Helm.

A pod is refused with unable to validate against any security context constraint

Section titled “A pod is refused with unable to validate against any security context constraint”

The pod pins a UID outside the namespace’s range. If it is an Omnia pod, check that the operator and arena controller run with --omit-pod-ids:

Terminal window
kubectl get deploy -n omnia-system -o yaml | grep omit-pod-ids

If it is a pod from podOverrides or extraObjects you added, remove its runAsUser, runAsGroup and fsGroup.

A sandbox or arena worker cannot write workspace content (permission denied)

Section titled “A sandbox or arena worker cannot write workspace content (permission denied)”

The workspace namespace has a different supplemental-group range from the release namespace. Align them as in step 4.

The install fails with openshift.enabled: nfs.csiDriver.enabled is not supported

Section titled “The install fails with openshift.enabled: nfs.csiDriver.enabled is not supported”

Set nfs.csiDriver.enabled: false and set workspaces.storage.storageClass to an RWX class the cluster provides.