QwenAudio/qwen-audio-agent

This project is a voice runtime for agents: it handles full-duplex speech in the foreground while pushing tool use, file work, and long-running jobs to a separate backend agent, then reports results back into the conversation. It suits people who want a persistent spoken assistant layered on top of an existing agent rather than a standalone voice app.

What it does

The runtime splits into two independently combined parts. A voice frontend handles realtime conversation, interruption, and multi-turn speech; a backend agent executes tasks. Answers that need no tools are spoken directly, while work requiring tools or sustained processing is delegated asynchronously, with progress follow-up and cancellation available at any time. Completed results return to the current conversation so you can ask follow-ups or request changes. It also documents per-user long-term memory across sessions.

How you set it up

Three interfaces are offered: WebUI, a terminal TUI, and a desktop floating orb for macOS, Windows, and Linux. Installation needs Node.js 22.22.2+ or 24.15.0+ and npm 10+, and the README recommends the global npm install; installation notes cover building from source.

A default sample config uses the DashScope Qwen Audio Realtime frontend, requiring a Bailian API key and pointing to the API key page. Frontend and backend are chosen separately. Frontends range from cloud services (Qwen Realtime, OpenAI Realtime, Gemini Live, Doubao, StepAudio) to local deployment (Hugging Face Speech-to-Speech, MiniCPM-o). Backends are mostly coding agents via ACP adapters — Qwen Code, OpenCode, OpenClaw, Codex, Claude Code and others — plus a frontend-only mode and an experimental bare-LLM backend. Integration ratings in the README distinguish thoroughly tested five-star links from those still under development or not fully verified; backend documentation covers each capability boundary.

Use paths and requirements

• Desktop app: floating orb with a built-in Gateway, idle sleep, and local wake word. Installers are on the releases page; building requires the platform-specific npm build scripts, and the Linux build is noted as unsigned (AppImage and deb).
• Gateway + TUI: run the Gateway in one terminal and qwenaudio tui in another, or use qwenaudio webui for the browser UI.
• Custom frontends: implement the Realtime Provider interface without touching core session or task logic.

The backend agent keeps its own model credentials and configuration, so reusing an agent means reusing its tools, MCP, Skills, and authentication. For the default cloud setup, quota and billing follow the current Bailian rules; the project links a new-user quota page. Data flow and privacy are covered separately in the project's PRIVACY.md.

Scope and caveats

The README states the current framework focuses on desktop productivity. Example scenarios include smart cockpit, customer service, and a hardware-card demo (half-duplex only); embodied intelligence and livestream assistance are listed as planned, not available.

Detailed integration constraints reside in per-frontend and per-backend docs rather than this overview, so the table above is a starting point, not a guarantee that every combination is tested. The repository is Apache-2.0 licensed.