How to Build a Voice Assistant with OpenClaw

Gartner projects $80 billion in contact center labor cost savings from conversational AI in 2026, driven largely by voice agents that handle calls at roughly $0.40 each compared to $7 to $12 for a human representative. The economics have flipped so far that the harder question is no longer whether to build a voice agent, but which framework handles the plumbing without requiring a custom pipeline from scratch.

OpenClaw answers that by treating voice as native I/O rather than a bolt-on integration. The platform ships with 14 TTS providers, multiple STT engines with automatic fallback detection, and a voice-call plugin that connects to phone networks through Twilio, Telnyx, and Plivo. Voice messages on Telegram, real-time conversations in Discord voice channels, and browser-based Talk Mode all run through the same underlying architecture.

The core loop is straightforward. A user speaks. A speech-to-text provider converts the audio to text. OpenClaw's agent processes the transcript through its LLM, skills, and tools. A text-to-speech provider converts the response back to audio. That loop runs identically whether the input arrives as a Telegram voice note, a Discord stream, a phone call, or a browser microphone capture.

This guide covers each piece of that pipeline: choosing STT and TTS providers, configuring them in OpenClaw, and deploying across the channels your users actually use.

Every voice interaction passes through five stages inside OpenClaw, regardless of which channel delivers the audio.

Stage 1: Voice Activity Detection. On real-time channels like Discord voice channels, Talk Mode, and phone calls, OpenClaw detects when someone starts and stops speaking. This filters background noise and determines when to capture audio for transcription. On asynchronous channels like Telegram, the platform receives a complete audio file and skips this stage.

Stage 2: Speech-to-Text. The captured audio goes to your configured STT provider. OpenClaw supports cloud providers (OpenAI Whisper API, Deepgram, ElevenLabs) and local options (Whisper CLI running on your own hardware). When no provider is explicitly set, the platform auto-detects available options in a fallback chain: local Whisper first, then cloud APIs in order.

Stage 3: Agent Processing. The transcript enters OpenClaw's standard agent loop. The LLM receives the text along with the agent's system prompt, conversation history, and any configured skills or tools. Processing is identical to a typed message. The agent reasons through the request and generates a text response.

Stage 4: Text-to-Speech. The text response goes to your configured TTS provider. OpenClaw selects the output format based on the delivery channel automatically. Messaging platforms that support voice notes (Telegram, WhatsApp) get Opus at 48kHz. Other channels receive MP3 at 44.1kHz. Telephony connections get raw PCM or ulaw at 8kHz.

Stage 5: Delivery. The audio response ships back through whatever channel initiated the conversation. On Discord, it plays through the voice channel. On Telegram, it arrives as a voice note. On the web, it plays through the browser.

Two audio processing modes exist, and they are mutually exclusive.

Streaming mode captures audio in real time and forwards chunks to an external STT provider for transcription, then passes the text through the agent and a separate TTS provider.

Realtime mode uses full-duplex voice models from OpenAI or Google Gemini Live that handle both transcription and response generation in a single connection. Realtime mode enables natural conversation with interruption support, but requires a compatible model.

Your STT provider determines how accurately and quickly your voice assistant transcribes spoken input. The choice affects latency, cost, language coverage, and whether you need an internet connection at all.

Local Whisper runs OpenAI's Whisper model on your own hardware. No API key needed, no per-request cost, and complete privacy since audio never leaves your machine. The tradeoff is compute: you need a capable GPU for real-time transcription, and CPU-only inference adds noticeable delay. This works well for personal assistants or development environments where you want to iterate without burning API credits.

OpenAI Whisper API sends audio to OpenAI's cloud endpoint and returns text. It handles 99+ languages with automatic detection, costs fractions of a cent per minute, and requires only an API key. Latency depends on audio length and network conditions, but it is fast enough for conversational use on asynchronous channels like Telegram.

Deepgram offers both batch and real-time streaming transcription. Streaming mode is particularly valuable for phone calls and Discord voice channels where you want word-by-word transcription as the speaker talks rather than waiting for them to finish. Deepgram also supports speaker diarization if you need to distinguish between multiple voices in the same audio stream.

ElevenLabs provides transcription alongside its TTS services. If you already use ElevenLabs for voice output, consolidating both STT and TTS under one provider simplifies billing and API key management.

When no provider is explicitly configured, OpenClaw walks a fallback chain: it checks for a local Whisper installation first, then tries cloud providers in order. For production deployments, set a provider explicitly rather than relying on auto-detection. Configure your preferred provider and its API credentials in your OpenClaw configuration file.

For real-time channels (phone calls, Discord voice, Talk Mode), prioritize providers that support streaming transcription. Batch providers still work on these channels, but they introduce a pause while the full audio chunk processes before the agent can respond.

TTS is what gives your voice assistant its spoken personality. OpenClaw supports 14 providers, and the configuration lives in your main OpenClaw config under the messages.tts section.

ElevenLabs produces the most natural-sounding speech among the available options and supports voice cloning, multilingual output, and deterministic generation via a seed parameter. It is the go-to choice when voice quality matters more than cost.

OpenAI TTS offers strong quality at lower cost, with persona instruction support that lets you describe the speaking style in natural language rather than selecting from a fixed voice menu.

Microsoft Edge TTS requires no API key and no payment. Quality is serviceable for development and testing, though it sounds noticeably more synthetic than ElevenLabs or OpenAI. This is a good starting point for prototyping before committing to a paid provider.

Azure Speech handles telephony-grade audio natively and outputs Ogg/Opus for messaging platforms. If you are building a phone-based assistant, Azure is worth evaluating for its format flexibility.

Google Gemini supports batch TTS with persona-aware prompt templates. If you already use Gemini as your LLM and want to keep your provider surface small, this consolidates one more dependency.

Configuration lives in your main OpenClaw config under the TTS section. You set the provider, supply API credentials, and choose whether TTS runs automatically on every response or only on demand. The OpenClaw TTS documentation covers the full configuration surface for each provider.

Per-Agent Voice Overrides

Different agents can speak with different voices. A customer support agent might use a warm, conversational tone while a data reporting agent uses a more neutral delivery. OpenClaw supports per-agent TTS overrides that deep-merge over the global config, so you only specify the settings that differ. Swap one voice ID and everything else inherits from the global defaults.

Personas

Personas are stable spoken identities that work across providers. A persona bundles a voice selection, speaking style, and provider-specific settings under a single name. You can define a "narrator" persona that uses ElevenLabs with a specific voice and style prompt, then switch the entire personality with one config change rather than updating individual provider settings.

OpenClaw includes built-in commands for testing TTS output without running through the full agent loop, so you can preview voice quality and verify provider connectivity before deploying to users.

Give your voice agent persistent storage and searchable transcripts

Fastio indexes transcripts for semantic search and RAG chat the moment they land in a workspace. generous storage, MCP server access, and ownership transfer when the project is ready for human handoff. No credit card required.

Start 14-Day Trial

With STT and TTS configured, the same voice pipeline works across every channel OpenClaw supports. Each channel handles audio capture and delivery differently, but the core loop (capture, transcribe, process, synthesize, deliver) stays the same.

Telegram Voice Messages

Telegram is the simplest starting point. When a user sends a voice message to your OpenClaw bot, Telegram delivers the audio file directly. OpenClaw transcribes it, processes the text through the agent, generates a TTS response, and sends it back as a voice note. No additional voice-specific configuration is needed beyond having STT and TTS enabled globally, since the existing Telegram bot integration handles routing automatically.

Discord Voice Channels

Discord support goes beyond asynchronous voice messages. OpenClaw can join Discord voice channels and participate in real-time spoken conversations with multiple users. The bot uses Voice Activity Detection to track who is speaking, transcribes in real time, and responds through the voice channel. It supports both a realtime conversation mode with natural turn-taking and interruption handling, and a legacy STT-TTS pipeline that works with any transcription provider.

The bot can join channels on command, connect automatically at startup, or follow specified users into channels they join. Grant it the standard voice and messaging permissions in your Discord server, and ensure your gateway host has audio codec support installed for format conversion. The OpenClaw Discord voice documentation covers the full setup.

Web Browser Talk Mode

Talk Mode turns your browser into a voice interface for one-on-one conversations. It uses the browser's built-in microphone for input, runs your configured STT provider for transcription, processes through the agent, and plays TTS output through browser audio. This mode is designed for personal assistants and internal tools where a direct spoken interface is faster than typing.

Phone Calls via the Voice-Call Plugin The voice-call plugin supports inbound and outbound phone calls through Twilio, Telnyx, or Plivo. This is the most involved channel to configure because it requires a publicly reachable automation hooks URL, a phone number from your carrier, and carrier-specific API credentials.

The plugin supports per-number routing, so different phone numbers can map to different agents or personas. Conversation history can persist across calls from the same number or reset per call, depending on your session scope setting. The plugin also handles DTMF tone input for IVR navigation and automated meeting joins.

A voice assistant generates artifacts that outlive any single conversation: transcripts, audio recordings, customer interaction logs, and structured data extracted from spoken requests. These need to live somewhere persistent, accessible to your team, and searchable without manually tagging files.

Local storage works during development but breaks down fast. Transcripts saved to a server's filesystem are invisible to everyone else, unsearchable, and gone if the instance restarts. Cloud storage services like S3 or Google Drive solve the persistence problem but add integration overhead and offer no built-in intelligence layer.

Fastio provides workspace storage designed for this kind of agent-to-human workflow. Your OpenClaw voice agent can write transcripts, audio files, and conversation summaries to a Fastio workspace through the MCP server or REST API. Once files land in a workspace with Intelligence enabled, they are automatically indexed for semantic search and RAG chat. A team member can ask "What did the caller say about the delivery timeline?" and get cited answers pulled directly from the transcript.

The ownership transfer feature fits voice assistant workflows well. An agent can build up a workspace of call recordings and transcripts during a project, then transfer the entire organization to a human stakeholder when the engagement ends. The agent retains admin access for ongoing maintenance while the human owns the data and controls sharing.

For structured extraction from voice interactions, Metadata Views can turn transcripts into a queryable database. Describe the fields you want (caller name, request type, action items, sentiment) in natural language, and the system populates a sortable spreadsheet from your transcript files without OCR rules or manual data entry.

The Business Trial includes 50 GB of storage, included credits, and 5 workspaces with no credit card and no expiration. For comparison, a 30-minute phone call transcript runs roughly 5,000 words, so the free tier covers thousands of conversations before storage becomes a consideration.