# Provider CRD

The Provider custom resource defines a reusable LLM provider configuration that can be referenced by multiple AgentRuntimes. This enables centralized credential management and consistent model configuration across agents.

## API version

```yaml
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
```

**Short name:** `prov` (e.g. `kubectl get prov`).

## Spec fields

### `type`

The provider wire protocol (message/response format).

| Value | Description | Requires Credentials |
|-------|-------------|----------------------|
| `claude` | Anthropic Claude wire protocol | Yes (unless hosted via `platform`) |
| `openai` | OpenAI chat completions wire protocol | Yes (unless hosted via `platform`) |
| `gemini` | Google Gemini wire protocol | Yes (unless hosted via `platform`) |
| `vllm` | vLLM-served OpenAI-compatible endpoint (set `baseURL`) | No (auth via custom `headers`) |
| `voyageai` | Voyage AI — `role: embedding` or `rerank` only | Yes (`VOYAGE_API_KEY`) |
| `cohere` | Cohere rerank — `role: rerank` only | Yes (`COHERE_API_KEY`) |
| `cartesia` | Cartesia text-to-speech — `role: tts` only | Yes |
| `elevenlabs` | ElevenLabs text-to-speech — `role: tts` only | Yes |
| `imagen` | Google Imagen image generation — `role: image` only | Yes |
| `huggingface` | HuggingFace Inference API / Endpoints — `role: inference` only | Yes |
| `ollama` | Local Ollama models (for development) | No |
| `mock` | Mock provider (for testing) | No |

```yaml
spec:
  type: claude
```

> **Hyperscaler hosting** (AWS Bedrock, Azure AI Foundry, GCP Vertex AI) is expressed by setting `spec.platform` (and `spec.auth`) on a `claude`, `openai`, or `gemini` provider — not as a separate provider type. See [`platform`](#platform) below.

### `role`

What kind of provider this is. Defaults to `llm`. An AgentRuntime's
`providers[].role` (also defaulting to `llm`) must match the referenced
Provider's role, or the AgentRuntime reports `ProviderReady=False` (reason
`RoleMismatch`) and `phase: Failed`.

Each role admits only certain types (enforced by CEL at admission):

| `role` | Allowed `type` |
|--------|----------------|
| `llm` | `claude`, `openai`, `gemini`, `ollama`, `mock`, `vllm` |
| `embedding` | `openai`, `voyageai`, `gemini`, `ollama` |
| `rerank` | `cohere`, `voyageai`, `mock` |
| `tts` | `openai`, `cartesia`, `elevenlabs` |
| `stt` | `openai` |
| `image` | `imagen` |
| `inference` | `huggingface` |

The `rerank` role is reserved. The CRD accepts it, but no Omnia component
consumes a rerank Provider: memory retrieval does not rerank, and a MemoryPolicy
or Workspace has no field that names one.

`spec.embedding` is a role-specific block, accepted only when `role` is
`embedding`. See [Memory](/explanation/agents/memory/#embedding-providers)
for how embedding providers back memory recall.

`spec.embedding.dimensions` (1 to 4096) is the vector size to request. memory-api
passes it to PromptKit, which sends it as `dimensions` for `openai` and `ollama`,
as `output_dimension` for `voyageai` and as `outputDimensionality` for `gemini`.
A model that cannot shorten its output returns its native size, and PromptKit
reports a dimension mismatch. memory-api refuses to start with a value above
4000, the largest its pgvector columns accept.

`spec.stt` and `spec.tts` are the role-specific blocks for `stt` and `tts`.
The runtime reads them for an agent's cascaded voice calls
([`spec.duplex.mode: cascaded`](/how-to/agents/add-voice-to-a-text-only-agent/)):

| Field | Effect |
|-------|--------|
| `stt.language` | Transcription language, as an ISO 639-1 code. Unset means `en` |
| `stt.sampleRate` | Input sample rate (8000 to 48000) the client is asked to capture at, unless the AgentRuntime's `spec.duplex.audio.recommendedSampleRate` sets one |
| `tts.voice` | Voice of the spoken reply. Unset means `alloy` |
| `tts.sampleRate`, `tts.format` | Not applied: replies are 16-bit PCM at 24 kHz. The runtime logs `tts output format not applied in cascaded voice` when either is set |

### `model`

The model identifier to use. Required for every provider type except `mock`. Omnia
applies no default: an empty model reaches the vendor API and is rejected at the first
request. A model-less non-mock Provider is rejected at admission; a Provider that
predates that rule reports `ModelValid=False` and `phase: Error`.

| Provider | Example Models |
|----------|----------------|
| Claude (direct) | `claude-sonnet-4-20250514`, `claude-opus-4-20250514` |
| Claude on Bedrock | `claude-sonnet-4-20250514` — auto-mapped to the Bedrock model ID |
| OpenAI (direct) | `gpt-4o`, `gpt-4-turbo`, `gpt-3.5-turbo` |
| OpenAI on Azure | Your Azure deployment name (e.g., `gpt-4o`) |
| Gemini (direct) | `gemini-pro`, `gemini-1.5-pro` |
| Gemini on Vertex | `gemini-1.5-pro`, `gemini-1.5-flash` |
| vLLM / Ollama | Model name served by the endpoint (e.g., `llama3:8b`) |

```yaml
spec:
  type: claude
  model: claude-sonnet-4-20250514
```

### `credential`

Provider credential configuration. Exactly one sub-field must be specified (or omit `credential` entirely for providers that don't need credentials, e.g. `mock`, `ollama`, `vllm`).

| Field | Type | Description |
|-------|------|-------------|
| `credential.secretRef.name` | string | Name of a Kubernetes Secret |
| `credential.secretRef.key` | string | Specific key within the Secret (auto-detected if omitted) |
| `credential.envVar` | string | Environment variable name containing the credential |
| `credential.filePath` | string | Path to a file containing the credential |

#### Using a Kubernetes secret

Reference to a Kubernetes Secret containing the credential value:

```yaml
spec:
  credential:
    secretRef:
      name: anthropic-credentials
      key: ANTHROPIC_API_KEY  # optional
```

When `key` is omitted, the first of these keys the Secret holds is used. The Provider's status, the admission webhook and every component that reads the credential (runtime, memory-api, eval worker, arena worker) make the same choice, so a Provider reported `CredentialConfigured` is one they can read. The condition's message names the key it found.

| Type | Keys tried, in order |
|------|----------------------|
| `claude` | `ANTHROPIC_API_KEY`, `CLAUDE_API_KEY`, `api-key` |
| `openai` | `OPENAI_API_KEY`, `OPENAI_TOKEN`, `api-key` |
| `gemini` | `GEMINI_API_KEY`, `GOOGLE_API_KEY`, `api-key` |
| `voyageai` | `VOYAGE_API_KEY`, `api-key` |
| `cohere` | `COHERE_API_KEY`, `api-key` |
| `huggingface` | `HF_TOKEN`, `api-key` |
| any other type (`ollama`, `vllm`, `mock`, `cartesia`, `elevenlabs`, `imagen`) | `api-key` only — another vendor's key is never sent to this provider |

An explicit `key` is used exactly: if the Secret lacks it, no other key is tried.

#### Using an environment variable

Typical use: CI/CD pipelines or environments where credentials are pre-injected as environment variables.

```yaml
spec:
  credential:
    envVar: ANTHROPIC_API_KEY
```

The variable must be available in the pod that consumes the provider (the runtime, memory-api or the eval worker, depending on which resolves it). That pod reads the value at resolution time.

The operator validates the variable's name and sets `CredentialConfigured` with reason `EnvVarConfigured`. It cannot see the pod's environment, so `CredentialValid` is `Unknown` with reason `ValidationNotSupported`.

A variable that is missing or empty is an error at resolution, not an absent credential: the request fails before it reaches the model provider, instead of going out unauthenticated.

#### Using a mounted file

Typical use: credentials mounted as files via a volume mount or CSI driver.

```yaml
spec:
  credential:
    filePath: /var/secrets/api-key
```

The file must be mounted in the pod that consumes the provider, which reads it at resolution time. Trailing whitespace is trimmed: a credential file written by an editor or projected by a CSI driver usually ends in a newline, and a bearer token carrying `\n` fails in a way that looks like a wrong key.

As with `envVar`, the operator validates the path (absolute and clean) and sets `CredentialConfigured` with reason `FilePathConfigured`, but cannot see the pod's filesystem, so `CredentialValid` is `Unknown`. A file that cannot be read, or is empty, is an error at resolution rather than a silently unauthenticated request.

### `platform`

Hyperscaler hosting configuration. Valid only on `claude`, `openai` and `gemini` providers with `role` `llm` or `embedding`, and always set together with [`auth`](#auth). A platform-hosted provider needs no `credential`.

A platform-hosted `embedding` provider works both as an agent's default provider and as a non-default entry in an AgentRuntime's [`spec.providers`](/reference/core/agentruntime/#providers).

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `platform.type` | string | Yes | Hyperscaler hosting platform: `bedrock`, `vertex`, or `azure` |
| `platform.region` | string | No | Cloud region (e.g., `us-east-1`, `us-central1`). Set it for `bedrock` and `vertex`; for `azure` the region comes from the endpoint. |
| `platform.project` | string | Conditional | GCP project ID — required (CEL) when `platform.type` is `vertex`. |
| `platform.endpoint` | string | Conditional | Platform API endpoint — required (CEL) when `platform.type` is `azure` (the Azure OpenAI resource URL). |

**Provider × platform combinations:**

Six pairs of wire protocol (`spec.type`) and hosting platform (`spec.platform.type`) are supported: the CRD admits them and the PromptKit runtime (v1.4.6+) routes requests correctly.

| provider type | supported platforms        |
|---------------|----------------------------|
| `claude`      | `bedrock`, `vertex`, `azure` |
| `openai`      | `azure`, `bedrock`           |
| `gemini`      | `vertex`                     |

Three pairs are rejected at admission because the hyperscaler does not host that model vendor as a partner endpoint:

- `openai × vertex` — Vertex AI does not host OpenAI.
- `gemini × bedrock` — AWS Bedrock does not host Gemini.
- `gemini × azure` — Azure AI Foundry does not host Gemini.

Setting `spec.platform` on any other provider type (e.g., `vllm`, `ollama`, `mock`) is also rejected at admission. Setting `spec.platform` without `spec.auth` (or vice versa) is likewise rejected.

The authentication method is determined by the platform (see [`auth`](#auth) below):

| platform | allowed auth methods                    |
|----------|-----------------------------------------|
| bedrock  | `workloadIdentity`, `accessKey`         |
| vertex   | `workloadIdentity`, `serviceAccount`    |
| azure    | `workloadIdentity`, `servicePrincipal`  |

#### Claude on AWS Bedrock

```yaml
spec:
  type: claude
  model: claude-sonnet-4-20250514   # auto-mapped to the Bedrock model ID
  platform:
    type: bedrock
    region: us-east-1
```

#### Gemini on GCP Vertex AI

```yaml
spec:
  type: gemini
  model: gemini-1.5-pro
  platform:
    type: vertex
    region: us-central1
    project: my-gcp-project
```

#### OpenAI on Azure AI foundry

```yaml
spec:
  type: openai
  model: gpt-4o
  platform:
    type: azure
    endpoint: https://my-resource.openai.azure.com
```

### `auth`

Authentication configuration for platform-hosted providers. Required when `spec.platform` is set; forbidden otherwise.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `auth.type` | string | Yes | Authentication method (see matrix below) |
| `auth.credentialsSecretRef` | SecretKeyRef | No | Secret containing platform credentials (required for static auth) |

#### Platform × auth matrix (enforced by CEL)

| `platform.type` | allowed `auth.type` |
|-----------------|---------------------|
| `bedrock`       | `workloadIdentity`, `accessKey` |
| `vertex`        | `workloadIdentity`, `serviceAccount` |
| `azure`         | `workloadIdentity`, `servicePrincipal` |

**Expected secret keys per static auth type:**

| `auth.type`       | Required keys in `credentialsSecretRef` |
|-------------------|------------------------------------------|
| `accessKey`       | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (optional `AWS_SESSION_TOKEN`) |
| `serviceAccount`  | `credentials.json` (or a custom key set via `credentialsSecretRef.key`) |
| `servicePrincipal`| `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` |

**CEL rules:**
- `credentialsSecretRef` is **required** for `accessKey`, `serviceAccount`, and `servicePrincipal`.
- `credentialsSecretRef` is **disallowed** for `workloadIdentity` (workload identity relies on pod-level identity).

#### Workload identity (recommended)

```yaml
spec:
  type: claude
  platform:
    type: bedrock
    region: us-east-1
  auth:
    type: workloadIdentity
```

Workload identity uses the agent pod's own identity. Bind the IAM role or
cloud service account to the pod's ServiceAccount; the Provider does not name
it.

#### Access key

```yaml
spec:
  type: claude
  platform:
    type: bedrock
    region: us-east-1
  auth:
    type: accessKey
    credentialsSecretRef:
      name: aws-credentials
```

### `baseURL`

Override the provider's default API endpoint. Useful for proxies, gateways (OpenRouter), or self-hosted models.

```yaml
spec:
  type: openai
  baseURL: https://my-openai-proxy.internal/v1
```

### `headers`

Custom HTTP headers included on every provider request. Typical use: gateway providers that require attribution headers (OpenRouter's `HTTP-Referer` and `X-Title`), or shared vLLM deployments that use tenant routing headers.

```yaml
spec:
  type: openai
  baseURL: https://openrouter.ai/api/v1
  headers:
    HTTP-Referer: https://example.com
    X-Title: omnia
  credential:
    secretRef:
      name: openrouter-credentials
```

Collisions with built-in provider headers are rejected by PromptKit at request time.

### `messageMetadataTemplate`

A text template that PromptKit renders from the application metadata attached to a user message. The rendered text is appended to that message's text in the request to the model. The stored conversation keeps the message as it was sent. Leave it unset and the model does not see the metadata.

```yaml
spec:
  type: claude
  model: claude-sonnet-4-20250514
  messageMetadataTemplate: "<context>Selected: {{selection}}</context>"
```

Each `{{key}}` placeholder is filled from the message's metadata:

| Metadata value | Rendered as |
|----------------|-------------|
| A string | The string, unchanged |
| Any other value | Compact JSON, for example `["s1","s2"]` |
| Missing or `null` | Empty |

A message whose template renders to empty text is sent unchanged.

Only valid when `spec.role` is `llm`. Admission rejects it on any other role, because PromptKit renders it on completion requests only. An agent applies it through its default provider. Providers listed in `spec.providers[]` for other roles, evals and Arena runs do not use it. Editing it rolls the agent's pods.

The metadata is the `metadata` map of the WebSocket `message`, or of the gRPC `ClientMessage` for a custom facade. The runtime attaches it to the user message with PromptKit's `sdk.WithMessageMetadata` only when this field is set, so a Provider without a template changes nothing. Reserved keys are never forwarded, forwarded metadata is size-limited, and it is recorded with the session as a `message.metadata` runtime event under the session's privacy policy. The [WebSocket protocol](/reference/platform/websocket-protocol/#message-metadata) lists the rules.

### `capabilities`

Lists what modalities and features this provider supports. An AgentRuntime binding can require capabilities with `providers[].requiredCapabilities`; when the Provider does not list every one, the AgentRuntime stays `Pending` with `ProviderReady=False` (reason `CapabilityMismatch`). The field is optional.

| Value | Description |
|-------|-------------|
| `text` | Text generation |
| `streaming` | Streaming responses |
| `vision` | Image/visual input |
| `tools` | Tool/function calling |
| `json` | Structured JSON output |
| `audio` | Audio input/output |
| `video` | Video input |
| `documents` | Document (PDF) input |
| `duplex` | Full-duplex (bidirectional streaming) |

```yaml
spec:
  capabilities:
    - text
    - streaming
    - vision
    - tools
    - json
```

### `defaults`

Tuning parameters applied to all requests using this provider. A non-zero `temperature` or `maxTokens` is sent on every request whose PromptPack prompt does not set that parameter itself; a prompt's own `parameters` take precedence. When neither sets it, an agent runs at `temperature` 0.7 and sends no output-token limit, so the model's own maximum applies; `claude`, whose API requires a limit, sends 4096.

| Field | Type | Range | Description |
|-------|------|-------|-------------|
| `temperature` | string | 0.0-2.0 | Controls randomness (lower = more focused) |
| `topP` | string | 0.0-1.0 | Nucleus sampling threshold |
| `maxTokens` | integer | - | Maximum tokens in response |
| `contextWindow` | integer | - | Model's maximum context size in tokens. When conversation history exceeds this budget, truncation is applied. If not specified, no automatic truncation is performed. |
| `truncationStrategy` | string | - | How to handle context overflow: `sliding` (default — remove oldest messages first), `summarize` (summarize old messages before removing), `custom` (the Omnia runtime applies no truncation, for a custom runtime that implements its own; the stock runtime logs a warning at startup) |
| `thinkingBudgetTokens` | integer | ≥ 1024 | Enables extended thinking and caps the reasoning tokens. Unset means thinking is off. Honoured by `claude`. The reasoning is recorded on the session alongside other events. |
| `requestTimeout` | string | - | Go duration capping non-streaming provider calls (predict, embeddings). Unset uses the provider's built-in default. |
| `streamIdleTimeout` | string | - | Go duration a streaming response may stay silent between reads before it is aborted. Defaults to `30s`. |
| `firstTokenTimeout` | string | - | Go duration a turn may go with no activity from the provider, chiefly the wait for its first token, before it ends with `PROVIDER_TIMEOUT`. Time inside a tool call does not count. Defaults to `90s`. |

```yaml
spec:
  defaults:
    temperature: "0.7"
    topP: "0.9"
    maxTokens: 4096
    contextWindow: 200000
    truncationStrategy: sliding
```

### `pricing`

Custom pricing for cost tracking. If not specified, PromptKit's built-in pricing is used.

| Field | Type | Description |
|-------|------|-------------|
| `inputCostPer1K` | string | Cost per 1000 input tokens |
| `outputCostPer1K` | string | Cost per 1000 output tokens |
| `cachedCostPer1K` | string | Cost per 1000 prompt-cache read tokens. Applied only when `inputCostPer1K` and `outputCostPer1K` are also set; without it, CRD pricing leaves cache reads unpriced. Prompt-cache writes have no rate and are not costed under CRD pricing. |

```yaml
spec:
  pricing:
    inputCostPer1K: "0.003"
    outputCostPer1K: "0.015"
```

## Status fields

### `phase`

| Value | Description |
|-------|-------------|
| `Ready` | Provider is usable: credentials are valid **and** a model is set |
| `Error` | Configuration error, invalid credentials, or a missing model |
| `Unavailable` | The endpoint health check has failed for more than 5 minutes (connection error, DNS failure or timeout; any HTTP response counts as reachable). The controller re-checks every 30 seconds. During the first 5 minutes of failures the phase stays `Ready`, matching the `Ready` condition, which then has reason `EndpointProbeFailing`. |

`Ready` means usable. A Provider with valid credentials but an empty `model`
reports `Error` (condition `ModelValid=False`) instead of `Ready`: an
AgentRuntime bound to it stays gated instead of appearing healthy and
failing mid-conversation with a 400.

### `conditions`

| Type | Description |
|------|-------------|
| `Ready` | Roll-up of `CredentialValid`, `CredentialConfigured` and `ModelValid`: `False` with the first failing condition's reason. Also `False` once `EndpointReachable` has been `False` for more than 5 minutes. Before then it stays `True` with reason `EndpointProbeFailing` and a message naming the failure |
| `SecretFound` | Referenced Secret exists and contains required key |
| `CredentialConfigured` | Credential source is configured (secretRef, envVar, or filePath). `False` with reason `PlaceholderCredential` when the Secret holds a dev-sample placeholder value |
| `CredentialValid` | Result of probing the vendor with a `secretRef` credential: `True` (`CredentialAccepted`), `False` (`CredentialRejected`), or `Unknown` (`ValidationNotSupported` for envVar/filePath and types with no probe, `CredentialValidationError` for a transient failure) |
| `EndpointReachable` | Health check of `baseURL` (or the vendor's default endpoint). Not set for `mock`, platform-hosted providers, the `tts`/`stt`/`image` roles, or a type with no endpoint to probe |
| `AuthConfigured` | Auth configuration is valid (hyperscaler providers only) |
| `ModelValid` | `spec.model` is set for provider types that require one (all except `mock`). `False` with reason `ModelMissing` when empty, and the message suggests valid model IDs |

## Complete examples

### API key provider

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: anthropic-credentials
  namespace: agents
stringData:
  ANTHROPIC_API_KEY: "sk-ant-api03-..."
---
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
metadata:
  name: claude-production
  namespace: agents
spec:
  type: claude
  model: claude-sonnet-4-20250514

  credential:
    secretRef:
      name: anthropic-credentials

  capabilities:
    - text
    - streaming
    - vision
    - tools
    - json

  defaults:
    temperature: "0.7"
    maxTokens: 4096
    contextWindow: 200000
    truncationStrategy: sliding

  pricing:
    inputCostPer1K: "0.003"
    outputCostPer1K: "0.015"
```

### Claude on AWS Bedrock with workload identity

```yaml
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
metadata:
  name: bedrock-production
  namespace: agents
spec:
  type: claude
  model: claude-sonnet-4-20250514   # auto-mapped to the corresponding Bedrock model ID

  platform:
    type: bedrock
    region: us-east-1

  auth:
    type: workloadIdentity

  capabilities:
    - text
    - streaming
    - vision
    - tools

  defaults:
    temperature: "0.7"
    maxTokens: 4096
```

## Using provider in AgentRuntime

Reference a Provider from an AgentRuntime using `providerRef`:

```yaml
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: AgentRuntime
metadata:
  name: my-agent
spec:
  promptPackRef:
    name: my-prompts

  providers:
    - name: default
      providerRef:
        name: claude-production  # a Provider in the agent's own namespace

  facades:
    - type: websocket
      port: 8080
```

## Multiple providers

Separate Provider resources can target different use cases, for example a
stricter temperature for production than for development:

```yaml
# Production provider with Claude Sonnet
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
metadata:
  name: claude-production
spec:
  type: claude
  model: claude-sonnet-4-20250514
  credential:
    secretRef:
      name: prod-credentials
  defaults:
    temperature: "0.3"  # More deterministic
---
apiVersion: omnia.altairalabs.ai/v1alpha1
kind: Provider
metadata:
  name: claude-development
spec:
  type: claude
  model: claude-haiku-20250514
  credential:
    secretRef:
      name: dev-credentials
  defaults:
    temperature: "0.7"
```

## Cross-namespace references are refused

A Provider is only ever used from its own namespace. `providerRef.namespace` must be omitted or equal the referencing resource's own namespace. An AgentRuntime naming any other namespace sets `ProviderReady=False` with reason `ProviderCrossNamespace`, goes to phase `Failed`, and has its Deployments stopped. An ArenaJob naming one fails its run.

Nothing is shared across workspaces, and a Provider carries a credential: resolving another namespace's Provider would hand one workspace's agents another workspace's key. Two workspaces that use the same model each get their own Provider and Secret.

## Related guides

- [Memory](/explanation/agents/memory/#embedding-providers) — how embedding providers back memory recall
- [Change the memory embedding model](/how-to/memory/change-memory-embedding-model/)
- [Configure AWS Bedrock Provider](/how-to/providers/configure-bedrock-provider/)
- [Configure GCP Vertex AI Provider](/how-to/providers/configure-vertex-provider/)
- [Configure Azure AI Provider](/how-to/providers/configure-azure-ai-provider/)