Manage workspaces
Create a workspace for a team, grant people and pipelines access to it, and deploy agents into its namespace.
Prerequisites
Section titled “Prerequisites”- The Omnia operator is installed.
kubectlaccess to the cluster.- For production, an identity provider that your dashboard authentication is set up against.
Create a workspace
Section titled “Create a workspace”-
Write a minimal Workspace. The controller creates the namespace because
createistrue.apiVersion: omnia.altairalabs.ai/v1alpha1kind: Workspacemetadata:name: my-teamspec:displayName: "My Team"namespace:name: omnia-my-teamcreate: true -
Apply it.
Terminal window kubectl apply -f workspace.yamlThe controller creates the namespace, one ServiceAccount per role (owner, editor, viewer) and the RBAC bindings for them.
-
Add a service group before you run real agents. Without one, agents in the workspace record no sessions, so token usage, cost and memory are unavailable. See Configure workspace service groups.
Configure access control
Section titled “Configure access control”The dashboard shows a user only the workspaces on which they resolve to a role. A workspace with no roleBindings, directGrants or anonymousAccess reaches Ready but is visible to nobody. A dashboard user’s role is the highest of the roleBindings[].groups entries that match their groups, an unexpired directGrants entry and anonymousAccess. anonymousAccess is a floor for every signed-in user as well as the role of unauthenticated visitors. ServiceAccount entries in roleBindings give a pipeline Kubernetes access and no dashboard role. See the Workspace reference for how roles resolve.
-
Map IdP groups to roles. A group name must equal the name in the user’s group claim exactly, including case.
spec:roleBindings:- groups:- "cs-admins@acme.com"role: owner- groups:- "cs-engineers@acme.com"- "support-team"role: editor- groups:- "cs-contractors@acme.com"role: viewer -
Optional: give a CI/CD ServiceAccount a role. The controller creates a Kubernetes RoleBinding in the workspace namespace for each ServiceAccount, so the pipeline can call the Kubernetes API directly.
spec:roleBindings:- serviceAccounts:- name: argocd-application-controllernamespace: argocdrole: editor -
Optional: grant one user a role for a limited time. Use groups for lasting access. The expiry below is illustrative.
spec:directGrants:- user: oncall@acme.comrole: ownerexpires: "2027-01-01T00:00:00Z" -
Optional, development only: give unauthenticated visitors a role, and every signed-in user at least that role. Use this when the dashboard runs in anonymous mode.
spec:anonymousAccess:enabled: truerole: viewerNever set
editororownerin production: anyone who reaches the dashboard, signed in or not, gets that role.
Limit resource usage
Section titled “Limit resource usage”-
Set
spec.quotason the Workspace. The controller writes it into aResourceQuotanamedworkspace-<workspace-name>-quotain the workspace namespace.spec:quotas:compute:requests:cpu: "50"memory: 100Giobjects:pods: 200persistentVolumeClaims: 20 -
Check that the quota is in force.
Terminal window kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="ResourceQuotaReady")]}'kubectl describe resourcequota workspace-my-team-quota -n omnia-my-teamResourceQuotaReady=Falsewith reasonResourceQuotaFailedcarries the API server’s error in its message, and the workspace phase isDegradeduntil it clears. ReasonResourceQuotaConflictmeans a ResourceQuota of that name, not made by the operator, is already in the namespace. The operator leaves it alone, so delete or rename it. -
Give every SandboxClass used in the workspace
resources.requests(or limits, which Kubernetes copies to requests). Under a compute quota the API server rejects a pod whose containers have no request for a capped resource. Omnia’s agent and service containers carry default requests.
To remove the quota, delete spec.quotas. The controller deletes the ResourceQuota.
status.quotas.resourceQuota on the Workspace shows the limits and what the namespace uses against them:
kubectl get workspace my-team -o jsonpath='{.status.quotas.resourceQuota}'Limit the agents in a workspace
Section titled “Limit the agents in a workspace”-
Set
spec.quotas.agentson the Workspace.spec:quotas:agents:maxAgentRuntimes: 20maxReplicasPerAgent: 5maxAgentRuntimescaps how many AgentRuntimes are admitted, oldest first.maxReplicasPerAgentcaps how many replicas each one runs, autoscaled or not. -
Check the usage.
Terminal window kubectl get workspace my-team -o jsonpath='{.status.quotas.agents}'kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="AgentQuotaWithinLimits")]}'admittedlists the AgentRuntimes that may run,waitingthose still waiting for admission, andclampedthose running at the replica cap.
An AgentRuntime the count limit has not admitted is not started: it stays Pending with WithinWorkspaceQuota=False, and starts once an older one is deleted or the limit is raised. Lowering maxAgentRuntimes stops nothing that is already admitted.
An AgentRuntime that asks for more replicas than maxReplicasPerAgent, through spec.runtime.replicas or an autoscaler’s maxReplicas, runs at the cap. Lowering the cap scales running agents down to it.
With the admission webhook enabled (webhook.enabled), the API server also refuses a create beyond maxAgentRuntimes, and an explicit spec.runtime.replicas above maxReplicasPerAgent.
Set a budget
Section titled “Set a budget”-
Set
spec.costControlson the Workspace. The workspace needs a service group with a session-api, because spend is read from it.spec:costControls:dailyBudget: "100.00" # USD, per UTC daymonthlyBudget: "2000.00" # USD, per UTC calendar monthbudgetExceededAction: pauseJobs # warn | pauseJobs | blockalertThresholds:- percent: 80- percent: 95Spend counts every provider call the workspace’s agents, try-it sessions and Arena runs made, plus session-less usage such as embeddings, across every service group. Service groups that share a database are counted once only when they name the same database Secret or session-api URL. Two Secrets pointing at one database double the spend, so give such groups one Secret.
-
Check the spend. The operator reads it about every five minutes.
Terminal window kubectl get workspace my-team -o jsonpath='{.status.costUsage}'kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="WithinBudget")]}'dailyPercentandmonthlyPercentare the spend as a percent of each budget.
When the day’s or the month’s spend reaches its budget, WithinBudget turns False and the operator emits a Warning event. What else happens depends on budgetExceededAction:
warndoes nothing more.pauseJobsholds every ArenaRun that has not started. It staysPendingwith reasonWorkspaceBudgetExceeded. Runs already running finish. Scheduled ArenaJobs skip their ticks. A held run takes no arena quota slot and is not counted againstmaxJobsPerDay; when the hold lifts, waiting runs are admitted oldest first.blockholds ArenaRuns too, and the agents refuse new work. A WebSocket client gets the error codeBUDGET_EXCEEDED, an A2A message fails, a function call gets429 budget_exceeded, and an MCP tool call gets abudget_exceededtool error. Open conversations carry on. Each agent pod reads its Workspace at most every 30 seconds, and admits all new work when it cannot read it, for example when it has no read permission on the Workspace or the API server is down.
The hold lifts at the next UTC midnight for a daily budget, or the first of the next month for a monthly one, or as soon as you raise the budget.
Each alert threshold emits one CostAlertThreshold Warning event per period, the first time spend reaches it. Alerts are at most once: an event lost to a failed status write is not repeated. The notify addresses are not emailed.
kubectl get events --field-selector involvedObject.name=my-team,reason=CostAlertThresholdLimit the Arena runs in a workspace
Section titled “Limit the Arena runs in a workspace”Arena is Enterprise: the arena controller enforces spec.quotas.arena, and a Core install ignores it.
-
Set
spec.quotas.arenaon the Workspace.spec:quotas:arena:maxConcurrentJobs: 3maxJobsPerDay: 50maxWorkersPerJob: 4Every limit counts ArenaRuns, the executions of an ArenaJob, not ArenaJobs. A manual ArenaJob creates one run; a scheduled one creates a run at every firing.
maxConcurrentJobscaps how many runs are admitted and unfinished at once. A run holds its slot from admission until it isSucceeded,FailedorCancelled. A run that has not started its workers 15 minutes after admission gives its slot back and waits again, behind the other waiting runs.maxJobsPerDaycaps how many runs are admitted in any rolling 24 hours, not per calendar day, up to 1000. A run is charged once, when it is admitted, however long it runs.maxWorkersPerJobcaps the worker replicas of one run.
-
Check the usage.
Terminal window kubectl get workspace my-team -o jsonpath='{.status.quotas.arena}'kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="ArenaQuotaWithinLimits")]}'runningandstartedLast24hare the counts the limits apply to.admittedlists the unfinished runs the quota has admitted,waitingthose still waiting, in the order they will be admitted,stalledthose whose admission was released because they did not start, andclampedthose running with fewer workers than they ask for.startsis the record of the last 24 hours of admissions thatmaxJobsPerDayis counted from. It is kept on the Workspace, so a controller restart, or an ArenaJob pruning its finished runs, does not reset the count. RemovingmaxJobsPerDaydiscards it.
No limit fails or stops a run. A run beyond maxConcurrentJobs or maxJobsPerDay waits, Pending with WithinWorkspaceQuota=False. Waiting runs are admitted oldest first, when an earlier run finishes or when the oldest admission is 24 hours old. A scheduled ArenaJob is the exception: when the runs already started or waiting use up the day’s allowance, it skips that firing with a RunSkipped event rather than adding a run that could not start until the window moves on.
A run asking for more workers than maxWorkersPerJob runs at the cap, and its status.workers shows the count it ran with. The cap applies when the run starts, so lowering it never shrinks a running run. Lowering maxConcurrentJobs or maxJobsPerDay stops nothing that is already admitted.
With the arena controller’s licence webhooks enabled, the API server also refuses an ArenaJob whose explicit spec.workers.replicas is above maxWorkersPerJob.
Set the environment and cost tags
Section titled “Set the environment and cost tags”Set the environment tier and the tags in the Workspace spec. The controller writes both onto the workspace namespace as labels.
spec: environment: production # development | staging | production defaultTags: team: "customer-support" cost-center: "CC-1234"Isolate the workspace network
Section titled “Isolate the workspace network”To restrict traffic to and from the namespace, set spec.networkPolicy.isolate: true. See Configure workspace network isolation.
Deploy an agent into the workspace
Section titled “Deploy an agent into the workspace”Create the AgentRuntime in the workspace namespace and select a service group.
apiVersion: omnia.altairalabs.ai/v1alpha1kind: AgentRuntimemetadata: name: support-bot namespace: omnia-my-teamspec: serviceGroup: default promptPackRef: name: support-prompts providers: - name: default providerRef: name: claude-providerVerify
Section titled “Verify”-
Read the workspace status. Workspace is cluster-scoped, so there is no
-n.Terminal window kubectl get workspace my-team -o yamlstatus.phaseisReady,status.namespace.nameis the namespace, andstatus.serviceAccountsnamesworkspace-my-team-owner-sa,workspace-my-team-editor-saandworkspace-my-team-viewer-sa. -
Confirm management-api serves the workspace to a signed-in user. Open
/api/v1/workspaceson the dashboard URL. An empty{"workspaces":[],"count":0}means the user resolves to no role, so check the bindings above. -
Switch workspaces in the dashboard with the selector in the header, which reads “Select workspace” until you pick one. It lists only the workspaces you have a role on.
Troubleshooting
Section titled “Troubleshooting”The phase is Pending
Section titled “The phase is Pending”The workspace storage PVC is not bound yet. Read the StorageReady and Ready conditions.
kubectl describe workspace my-teamThe phase is Error
Section titled “The phase is Error”Read the condition whose status is False. NamespaceReady with reason NamespaceConflict means another Workspace owns the namespace, and a message that the namespace does not exist means spec.namespace.create is false. The operator logs name the failing step.
kubectl logs -n omnia-system deploy/omnia-controller-managerA user cannot see the workspace
Section titled “A user cannot see the workspace”- Read the groups in the user’s token and compare them with
roleBindings[].groups. They must match exactly. - Check whether
directGrantshas an unexpired entry for the user’s email or username. - Check whether
anonymousAccessis enabled, if you are on a development install.
A pipeline ServiceAccount is refused by the Kubernetes API
Section titled “A pipeline ServiceAccount is refused by the Kubernetes API”-
Confirm the RoleBinding exists in the workspace namespace.
Terminal window kubectl get rolebindings -n omnia-my-team -
Confirm the ServiceAccount name and namespace in
roleBindingsmatch the pipeline’s.
New resources are rejected for quota
Section titled “New resources are rejected for quota”kubectl describe resourcequota -n omnia-my-teamRaise the limit in spec.quotas (or spec.hard on a ResourceQuota of your own), or remove unused resources.
An agent stays Pending with WithinWorkspaceQuota=False
Section titled “An agent stays Pending with WithinWorkspaceQuota=False”kubectl get agentruntime support-bot -n omnia-my-team \ -o jsonpath='{.status.conditions[?(@.type=="WithinWorkspaceQuota")].message}'AgentRuntimeQuotaExceeded means the Workspace has not admitted it: maxAgentRuntimes older AgentRuntimes are admitted. Delete one or raise the limit, and the agent starts. ReplicasClamped does not hold an agent; it only says it runs at the cap.
An ArenaRun stays Pending with WorkspaceBudgetExceeded
Section titled “An ArenaRun stays Pending with WorkspaceBudgetExceeded”The workspace is over a budget and its budgetExceededAction is pauseJobs or block. Read the spend and the condition.
kubectl get workspace my-team -o jsonpath='{.status.costUsage}'kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="WithinBudget")].message}'Raise the budget or wait for the period to roll over, and the run starts within a minute.
The budget is not enforced
Section titled “The budget is not enforced”Read the CostUsageAvailable condition. NoSessionAPI means no service group has a session-api, so no spend is recorded. SessionAPIUnavailable means the last read failed; the operator log line workspace spend read failed carries the error. A WithinBudget reason of InvalidBudget means a budget is not a positive number.
An Arena run stays Pending with WithinWorkspaceQuota=False
Section titled “An Arena run stays Pending with WithinWorkspaceQuota=False”kubectl get arenarun nightly-eval-x7k2p -n omnia-my-team \ -o jsonpath='{.status.conditions[?(@.type=="WithinWorkspaceQuota")].message}'WaitingForConcurrentRunSlot means maxConcurrentJobs runs are running: the run starts when one of them finishes. WaitingForDailyRunQuota means maxJobsPerDay runs were admitted in the last 24 hours: the message gives the time the next slot opens. WaitingForQuotaAdmission means the quota has not counted the run yet, which takes a moment after it is created. AdmissionReleased means the run was admitted but did not start its workers within 15 minutes, so it gave its slot back: read its other conditions (SourceValid, JobCreated) for why, and fix that. Cancel the run (spec.cancelled: true) to take it out of line. WorkersClamped does not hold a run; it only says the run executes at the worker cap.
For network, content, project and skill problems, see Configure workspace network isolation and Troubleshoot workspace content.
Related pages
Section titled “Related pages”- Configure workspace service groups
- Workspace reference: every field, status value and the complete example
- Multi-tenancy architecture
- Configure dashboard authentication