Add voice to a text-only agent

Run a voice call with an agent whose model has no native audio streaming, such as Claude. The runtime transcribes the caller’s speech, sends the text to the model, and speaks the reply.

Prerequisites

• An AgentRuntime whose default provider is a text model.
• A Provider with role: stt, such as OpenAI whisper-1.
• A Provider with role: tts, such as OpenAI tts-1, Cartesia or ElevenLabs.
• The credentials for both providers, in the same form as any other provider. See Provider.

Steps

1. Create the speech providers. spec.stt and spec.tts set what the call uses:

   apiVersion: omnia.altairalabs.ai/v1alpha1
   kind: Provider
   metadata:
     name: openai-stt
   spec:
     type: openai
     role: stt
     model: whisper-1
     credential:
       secretRef:
         name: openai-credentials
     stt:
       language: en
       sampleRate: 16000
   ---
   apiVersion: omnia.altairalabs.ai/v1alpha1
   kind: Provider
   metadata:
     name: openai-tts
   spec:
     type: openai
     role: tts
     model: tts-1
     credential:
       secretRef:
         name: openai-credentials
     tts:
       voice: nova

   Field	Effect on the call
   stt.language	Transcription language. Unset means en
   stt.sampleRate	Sample rate the console captures at, unless spec.duplex.audio.recommendedSampleRate sets one
   tts.voice	Voice of the agent’s replies. Unset means alloy

   The runtime speaks replies as 16-bit PCM at 24 kHz. It logs tts output format not applied in cascaded voice when tts.sampleRate or tts.format is set.

2. Bind both providers to the agent and turn on duplex:

   apiVersion: omnia.altairalabs.ai/v1alpha1
   kind: AgentRuntime
   metadata:
     name: support-agent
   spec:
     providers:
       - name: default
         providerRef:
           name: claude-sonnet
       - name: speech-in
         role: stt
         providerRef:
           name: openai-stt
       - name: speech-out
         role: tts
         providerRef:
           name: openai-tts
     duplex:
       enabled: true
       mode: cascaded

   With mode unset, the runtime chooses cascaded voice when the default provider’s type cannot stream audio and both speech providers are usable. Claude, Ollama and vLLM qualify. An OpenAI or Gemini provider streams audio by type whatever its model, so an agent on a text model from either vendor sets mode: cascaded.

3. Apply both manifests:

   kubectl --context <context> apply -f speech-providers.yaml -f support-agent.yaml

Verify

1. Open the agent’s console in the dashboard and start a call. Once the call connects, the console header shows Cascaded voice (STT → LLM → TTS). Its tooltip says that each turn adds latency. An agent on a realtime model shows Realtime voice instead. The label reflects the mode the runtime chose, including the automatic choice when mode is unset. A runtime that predates the label shows none.

2. Speak, then pause. The agent replies after about 800 ms of silence.

3. Check the runtime log for the session’s mode:

   kubectl --context <context> -n <namespace> logs deploy/support-agent -c runtime | grep "duplex mode selected"

   The line carries mode=cascaded.

Troubleshooting

• The call fails at open with spec.duplex.mode is cascaded but the agent has no usable stt and tts providers. One of the providers is missing from spec.providers, or the runtime skipped it. Search the runtime log for skipping extra provider, which names the provider and the missing credential.
• The agent answers slowly. Each turn waits for transcription, the full model reply and synthesis. A realtime model, such as Gemini Live, answers sooner.

Related

• AgentRuntime reference: duplex
• Provider reference