OpenBMB/VoxCPM

Overview

VoxCPM2 is a 2B-parameter text-to-speech model that generates continuous speech representations through a diffusion autoregressive architecture instead of discrete audio tokens. Built on a MiniCPM-4 backbone, it targets developers and researchers who need multilingual synthesis, voice creation from text descriptions, and cloning from reference audio.

The release supports 30 languages plus several Chinese dialects, outputs 48 kHz audio natively, and offers three cloning modes: voice design without reference audio, controllable cloning with style guidance, and "ultimate cloning" that also takes the reference transcript. Weight and code are released under Apache-2.0. The project also points to serving integrations on vLLM-Omni, Nano-vLLM, and llama.cpp-omni for throughput or on-device use.

Quick start

You need Python 3.10–3.12, PyTorch 2.5.0 or newer, and CUDA 12.0 or newer. Install the package and run the Python API:

pip install voxcpm

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2", load_denoiser=False)
wav = model.generate(
    text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)

This downloads the model, synthesizes one utterance, and writes demo.wav. The same flow works offline via ModelScope: download the snapshot into a local directory and pass that path to from_pretrained. A browser playground and audio samples are available for trying the model before installing it.

Usage modes

Voice design places a description in parentheses at the start of the text, for example (A young woman, gentle and sweet voice)Hello..., and needs no reference audio. Controllable cloning adds reference_wav_path while still allowing parenthetical style instructions. Ultimate cloning passes prompt_wav_path, prompt_text, and optionally the same file as reference_wav_path for maximum similarity.

The voxcpm CLI exposes design, clone, and batch subcommands, and a bundled web UI runs with python app.py --port 8808. generate_streaming returns chunks for low-latency delivery.

Using the model with other tools

For high-throughput serving, the README documents Nano-vLLM with an async API and vLLM-Omni with an OpenAI-compatible /v1/audio/speech endpoint and PagedAttention. On-device users can build llama.cpp-omni and run the voxcpm2-cli binary with GGUF weights on CPU, Metal, CUDA, or Vulkan. These are separate projects with their own setup steps; the README reports roughly 0.13 RTF on an RTX 4090 under Nano-vLLM and 1.76 RTF on an M4 Pro under llama.cpp, so plan hardware accordingly.

Constraints

• The standard PyTorch path requires CUDA 12.0+ and about 8 GB of VRAM for VoxCPM2; the web UI also accepts cpu, mps, or cuda:N via --device.
• Voice design and controllable cloning can vary between runs; the authors suggest generating one to three times to get the intended voice or style.
• Official support covers 30 languages; other languages require testing or fine-tuning on your own data.
• Fine-tuning supports full SFT and LoRA with 5–10 minutes of audio, but needs a separate training setup and data preparation.
• The authors forbid impersonation, fraud, and disinformation and recommend labeling AI-generated audio.
• Character-level timestamps are best-effort and derived from word alignment.

HotRepo 点评

VoxCPM 的日榜记录从 2026 年 9 月中旬延续到 11 月：9 月 14 日和 15 日分别新增约 204 和 216 Star，10 月 23 日新增 144 Star，11 月 9 日登上 Python 语言日榜第 6 名，当日新增 423 Star，总数达到 42,007。增速并非单调上升，但近两个月的持续上榜说明关注度稳定。项目 2025 年 9 月创建，以 2B 参数的开源权重和高星数进入 TTS 赛道，评测表已把 VoxCPM2 与闭源和多语言模型并列比较；对照时应以其自身公开基准和条件为准。

Project source and documentation