# Manage workspaces Create a workspace for a team, grant people and pipelines access to it, and deploy agents into its namespace. ## Prerequisites - The Omnia operator is installed. - `kubectl` access to the cluster. - For production, an identity provider that your [dashboard authentication](/how-to/security/configure-dashboard-auth/) is set up against. ## Create a workspace 1. Write a minimal Workspace. The controller creates the namespace because `create` is `true`. ```yaml apiVersion: omnia.altairalabs.ai/v1alpha1 kind: Workspace metadata: name: my-team spec: displayName: "My Team" namespace: name: omnia-my-team create: true ``` 2. Apply it. ```bash kubectl apply -f workspace.yaml ``` The controller creates the namespace, one ServiceAccount per role (owner, editor, viewer) and the RBAC bindings for them. 3. Add a service group before you run real agents. Without one, agents in the workspace record no sessions, so token usage, cost and memory are unavailable. See [Configure workspace service groups](/how-to/workspaces/configure-service-groups/). ## Configure access control The dashboard shows a user only the workspaces on which they resolve to a role. A workspace with no `roleBindings`, `directGrants` or `anonymousAccess` reaches `Ready` but is visible to nobody. A dashboard user's role is the highest of the `roleBindings[].groups` entries that match their groups, an unexpired `directGrants` entry and `anonymousAccess`. `anonymousAccess` is a floor for every signed-in user as well as the role of unauthenticated visitors. ServiceAccount entries in `roleBindings` give a pipeline Kubernetes access and no dashboard role. See the [Workspace reference](/reference/core/workspace/#rolebindings) for how roles resolve. 1. Map IdP groups to roles. A group name must equal the name in the user's group claim exactly, including case. ```yaml spec: roleBindings: - groups: - "cs-admins@acme.com" role: owner - groups: - "cs-engineers@acme.com" - "support-team" role: editor - groups: - "cs-contractors@acme.com" role: viewer ``` 2. Optional: give a CI/CD ServiceAccount a role. The controller creates a Kubernetes RoleBinding in the workspace namespace for each ServiceAccount, so the pipeline can call the Kubernetes API directly. ```yaml spec: roleBindings: - serviceAccounts: - name: argocd-application-controller namespace: argocd role: editor ``` 3. Optional: grant one user a role for a limited time. Use groups for lasting access. The expiry below is illustrative. ```yaml spec: directGrants: - user: oncall@acme.com role: owner expires: "2027-01-01T00:00:00Z" ``` 4. Optional, development only: give unauthenticated visitors a role, and every signed-in user at least that role. Use this when the dashboard runs in anonymous mode. ```yaml spec: anonymousAccess: enabled: true role: viewer ``` Never set `editor` or `owner` in production: anyone who reaches the dashboard, signed in or not, gets that role. ## Limit resource usage 1. Set `spec.quotas` on the Workspace. The controller writes it into a [`ResourceQuota`](https://kubernetes.io/docs/concepts/policy/resource-quotas/) named `workspace--quota` in the workspace namespace. ```yaml spec: quotas: compute: requests: cpu: "50" memory: 100Gi objects: pods: 200 persistentVolumeClaims: 20 ``` 2. Check that the quota is in force. ```bash kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="ResourceQuotaReady")]}' kubectl describe resourcequota workspace-my-team-quota -n omnia-my-team ``` `ResourceQuotaReady=False` with reason `ResourceQuotaFailed` carries the API server's error in its message, and the workspace phase is `Degraded` until it clears. Reason `ResourceQuotaConflict` means a ResourceQuota of that name, not made by the operator, is already in the namespace. The operator leaves it alone, so delete or rename it. 3. Give every SandboxClass used in the workspace `resources.requests` (or limits, which Kubernetes copies to requests). Under a compute quota the API server rejects a pod whose containers have no request for a capped resource. Omnia's agent and service containers carry default requests. To remove the quota, delete `spec.quotas`. The controller deletes the ResourceQuota. `status.quotas.resourceQuota` on the Workspace shows the limits and what the namespace uses against them: ```bash kubectl get workspace my-team -o jsonpath='{.status.quotas.resourceQuota}' ``` ## Limit the agents in a workspace 1. Set `spec.quotas.agents` on the Workspace. ```yaml spec: quotas: agents: maxAgentRuntimes: 20 maxReplicasPerAgent: 5 ``` `maxAgentRuntimes` caps how many AgentRuntimes are admitted, oldest first. `maxReplicasPerAgent` caps how many replicas each one runs, autoscaled or not. 2. Check the usage. ```bash kubectl get workspace my-team -o jsonpath='{.status.quotas.agents}' kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="AgentQuotaWithinLimits")]}' ``` `admitted` lists the AgentRuntimes that may run, `waiting` those still waiting for admission, and `clamped` those running at the replica cap. An AgentRuntime the count limit has not admitted is not started: it stays `Pending` with `WithinWorkspaceQuota=False`, and starts once an older one is deleted or the limit is raised. Lowering `maxAgentRuntimes` stops nothing that is already admitted. An AgentRuntime that asks for more replicas than `maxReplicasPerAgent`, through `spec.runtime.replicas` or an autoscaler's `maxReplicas`, runs at the cap. Lowering the cap scales running agents down to it. With the admission webhook enabled (`webhook.enabled`), the API server also refuses a create beyond `maxAgentRuntimes`, and an explicit `spec.runtime.replicas` above `maxReplicasPerAgent`. ## Set a budget 1. Set `spec.costControls` on the Workspace. The workspace needs a service group with a session-api, because spend is read from it. ```yaml spec: costControls: dailyBudget: "100.00" # USD, per UTC day monthlyBudget: "2000.00" # USD, per UTC calendar month budgetExceededAction: pauseJobs # warn | pauseJobs | block alertThresholds: - percent: 80 - percent: 95 ``` Spend counts every provider call the workspace's agents, try-it sessions and Arena runs made, plus session-less usage such as embeddings, across every service group. Service groups that share a database are counted once only when they name the same database Secret or session-api URL. Two Secrets pointing at one database double the spend, so give such groups one Secret. 2. Check the spend. The operator reads it about every five minutes. ```bash kubectl get workspace my-team -o jsonpath='{.status.costUsage}' kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="WithinBudget")]}' ``` `dailyPercent` and `monthlyPercent` are the spend as a percent of each budget. When the day's or the month's spend reaches its budget, `WithinBudget` turns `False` and the operator emits a Warning event. What else happens depends on `budgetExceededAction`: - `warn` does nothing more. - `pauseJobs` holds every ArenaRun that has not started. It stays `Pending` with reason `WorkspaceBudgetExceeded`. Runs already running finish. Scheduled ArenaJobs skip their ticks. A held run takes no arena quota slot and is not counted against `maxJobsPerDay`; when the hold lifts, waiting runs are admitted oldest first. - `block` holds ArenaRuns too, and the agents refuse new work. A WebSocket client gets the error code `BUDGET_EXCEEDED`, an A2A message fails, a function call gets `429 budget_exceeded`, and an MCP tool call gets a `budget_exceeded` tool error. Open conversations carry on. Each agent pod reads its Workspace at most every 30 seconds, and admits all new work when it cannot read it, for example when it has no read permission on the Workspace or the API server is down. The hold lifts at the next UTC midnight for a daily budget, or the first of the next month for a monthly one, or as soon as you raise the budget. Each alert threshold emits one `CostAlertThreshold` Warning event per period, the first time spend reaches it. Alerts are at most once: an event lost to a failed status write is not repeated. The `notify` addresses are not emailed. ```bash kubectl get events --field-selector involvedObject.name=my-team,reason=CostAlertThreshold ``` ## Limit the Arena runs in a workspace Arena is Enterprise: the arena controller enforces `spec.quotas.arena`, and a Core install ignores it. 1. Set `spec.quotas.arena` on the Workspace. ```yaml spec: quotas: arena: maxConcurrentJobs: 3 maxJobsPerDay: 50 maxWorkersPerJob: 4 ``` Every limit counts ArenaRuns, the executions of an ArenaJob, not ArenaJobs. A manual ArenaJob creates one run; a scheduled one creates a run at every firing. - `maxConcurrentJobs` caps how many runs are admitted and unfinished at once. A run holds its slot from admission until it is `Succeeded`, `Failed` or `Cancelled`. A run that has not started its workers 15 minutes after admission gives its slot back and waits again, behind the other waiting runs. - `maxJobsPerDay` caps how many runs are admitted in any rolling 24 hours, not per calendar day, up to 1000. A run is charged once, when it is admitted, however long it runs. - `maxWorkersPerJob` caps the worker replicas of one run. 2. Check the usage. ```bash kubectl get workspace my-team -o jsonpath='{.status.quotas.arena}' kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="ArenaQuotaWithinLimits")]}' ``` `running` and `startedLast24h` are the counts the limits apply to. `admitted` lists the unfinished runs the quota has admitted, `waiting` those still waiting, in the order they will be admitted, `stalled` those whose admission was released because they did not start, and `clamped` those running with fewer workers than they ask for. `starts` is the record of the last 24 hours of admissions that `maxJobsPerDay` is counted from. It is kept on the Workspace, so a controller restart, or an ArenaJob pruning its finished runs, does not reset the count. Removing `maxJobsPerDay` discards it. No limit fails or stops a run. A run beyond `maxConcurrentJobs` or `maxJobsPerDay` waits, `Pending` with `WithinWorkspaceQuota=False`. Waiting runs are admitted oldest first, when an earlier run finishes or when the oldest admission is 24 hours old. A scheduled ArenaJob is the exception: when the runs already started or waiting use up the day's allowance, it skips that firing with a `RunSkipped` event rather than adding a run that could not start until the window moves on. A run asking for more workers than `maxWorkersPerJob` runs at the cap, and its `status.workers` shows the count it ran with. The cap applies when the run starts, so lowering it never shrinks a running run. Lowering `maxConcurrentJobs` or `maxJobsPerDay` stops nothing that is already admitted. With the arena controller's licence webhooks enabled, the API server also refuses an ArenaJob whose explicit `spec.workers.replicas` is above `maxWorkersPerJob`. ## Set the environment and cost tags Set the environment tier and the tags in the Workspace spec. The controller writes both onto the workspace namespace as labels. ```yaml spec: environment: production # development | staging | production defaultTags: team: "customer-support" cost-center: "CC-1234" ``` ## Isolate the workspace network To restrict traffic to and from the namespace, set `spec.networkPolicy.isolate: true`. See [Configure workspace network isolation](/how-to/workspaces/configure-network-isolation/). ## Deploy an agent into the workspace Create the AgentRuntime in the workspace namespace and select a service group. ```yaml apiVersion: omnia.altairalabs.ai/v1alpha1 kind: AgentRuntime metadata: name: support-bot namespace: omnia-my-team spec: serviceGroup: default promptPackRef: name: support-prompts providers: - name: default providerRef: name: claude-provider ``` ## Verify 1. Read the workspace status. Workspace is cluster-scoped, so there is no `-n`. ```bash kubectl get workspace my-team -o yaml ``` `status.phase` is `Ready`, `status.namespace.name` is the namespace, and `status.serviceAccounts` names `workspace-my-team-owner-sa`, `workspace-my-team-editor-sa` and `workspace-my-team-viewer-sa`. 2. Confirm management-api serves the workspace to a signed-in user. Open `/api/v1/workspaces` on the dashboard URL. An empty `{"workspaces":[],"count":0}` means the user resolves to no role, so check the bindings above. 3. Switch workspaces in the dashboard with the selector in the header, which reads "Select workspace" until you pick one. It lists only the workspaces you have a role on. ## Troubleshooting ### The phase is Pending The workspace storage PVC is not bound yet. Read the `StorageReady` and `Ready` conditions. ```bash kubectl describe workspace my-team ``` ### The phase is Error Read the condition whose status is `False`. `NamespaceReady` with reason `NamespaceConflict` means another Workspace owns the namespace, and a message that the namespace does not exist means `spec.namespace.create` is `false`. The operator logs name the failing step. ```bash kubectl logs -n omnia-system deploy/omnia-controller-manager ``` ### A user cannot see the workspace 1. Read the groups in the user's token and compare them with `roleBindings[].groups`. They must match exactly. 2. Check whether `directGrants` has an unexpired entry for the user's email or username. 3. Check whether `anonymousAccess` is enabled, if you are on a development install. ### A pipeline ServiceAccount is refused by the Kubernetes API 1. Confirm the RoleBinding exists in the workspace namespace. ```bash kubectl get rolebindings -n omnia-my-team ``` 2. Confirm the ServiceAccount name and namespace in `roleBindings` match the pipeline's. ### New resources are rejected for quota ```bash kubectl describe resourcequota -n omnia-my-team ``` Raise the limit in `spec.quotas` (or `spec.hard` on a ResourceQuota of your own), or remove unused resources. ### An agent stays Pending with WithinWorkspaceQuota=False ```bash kubectl get agentruntime support-bot -n omnia-my-team \ -o jsonpath='{.status.conditions[?(@.type=="WithinWorkspaceQuota")].message}' ``` `AgentRuntimeQuotaExceeded` means the Workspace has not admitted it: `maxAgentRuntimes` older AgentRuntimes are admitted. Delete one or raise the limit, and the agent starts. `ReplicasClamped` does not hold an agent; it only says it runs at the cap. ### An ArenaRun stays Pending with WorkspaceBudgetExceeded The workspace is over a budget and its `budgetExceededAction` is `pauseJobs` or `block`. Read the spend and the condition. ```bash kubectl get workspace my-team -o jsonpath='{.status.costUsage}' kubectl get workspace my-team -o jsonpath='{.status.conditions[?(@.type=="WithinBudget")].message}' ``` Raise the budget or wait for the period to roll over, and the run starts within a minute. ### The budget is not enforced Read the `CostUsageAvailable` condition. `NoSessionAPI` means no service group has a session-api, so no spend is recorded. `SessionAPIUnavailable` means the last read failed; the operator log line `workspace spend read failed` carries the error. A `WithinBudget` reason of `InvalidBudget` means a budget is not a positive number. ### An Arena run stays Pending with WithinWorkspaceQuota=False ```bash kubectl get arenarun nightly-eval-x7k2p -n omnia-my-team \ -o jsonpath='{.status.conditions[?(@.type=="WithinWorkspaceQuota")].message}' ``` `WaitingForConcurrentRunSlot` means `maxConcurrentJobs` runs are running: the run starts when one of them finishes. `WaitingForDailyRunQuota` means `maxJobsPerDay` runs were admitted in the last 24 hours: the message gives the time the next slot opens. `WaitingForQuotaAdmission` means the quota has not counted the run yet, which takes a moment after it is created. `AdmissionReleased` means the run was admitted but did not start its workers within 15 minutes, so it gave its slot back: read its other conditions (`SourceValid`, `JobCreated`) for why, and fix that. Cancel the run (`spec.cancelled: true`) to take it out of line. `WorkersClamped` does not hold a run; it only says the run executes at the worker cap. For network, content, project and skill problems, see [Configure workspace network isolation](/how-to/workspaces/configure-network-isolation/#troubleshooting) and [Troubleshoot workspace content](/how-to/workspaces/troubleshoot-workspace-content/). ## Related pages - [Configure workspace service groups](/how-to/workspaces/configure-service-groups/) - [Workspace reference](/reference/core/workspace/): every field, status value and the complete example - [Multi-tenancy architecture](/explanation/platform/multi-tenancy/) - [Configure dashboard authentication](/how-to/security/configure-dashboard-auth/)