microsoft/VibeVoice

What it is

VibeVoice is Microsoft's family of open-weight voice models covering both speech recognition (ASR) and text-to-speech (TTS). The shared technical base is a pair of continuous speech tokenizers (acoustic and semantic) running at 7.5 Hz, combined with a next-token diffusion scheme where an LLM models textual and dialogue context and a diffusion head produces acoustic detail. The low frame rate is what makes hour-scale audio tractable.

The family currently spans:

• VibeVoice-ASR-7B — accepts up to 60 minutes of audio in a single pass (64K token context) and outputs structured transcription with speaker, timestamp, and content, plus user-supplied hotwords for domain terms. A streaming variant emits text chunk by chunk as audio arrives, and a BitNet build runs on CPU.
• VibeVoice-TTS-1.5B — long-form multi-speaker synthesis up to 90 minutes with up to 4 speakers (ICLR 2026 Oral).
• VibeVoice-Realtime-0.5B — lightweight streaming TTS with roughly 300 ms first-audio latency and around 10 minutes of robust generation.

It suits researchers and engineers who need long-form diarized transcription or podcast-style multi-speaker synthesis and are comfortable running models locally or via Hugging Face.

Getting started

The lowest-friction path is the hosted ASR playground at aka.ms/vibevoice-asr: upload or record audio and read the who/when/what transcript, with no local install. For TTS, the Realtime Colab notebook runs the 0.5B model end to end in the browser.

For local use, install the model from its Hugging Face repository — VibeVoice-ASR, VibeVoice-1.5B, or VibeVoice-Realtime-0.5B — and follow the matching documentation: ASR, TTS, Realtime. Faster ASR serving is documented for vLLM and vLLM streaming; fine-tuning guidance lives in the ASR fine-tuning README. CPU-only deployments can use the separate VibeASR.cpp engine with the BitNet weights.

The repository does not publish a single turnkey pip install + inference command in its README; the model-specific docs and Colab notebooks are the authoritative entry points.

Constraints and risks

• License is MIT, but the README frames the models as research and development artifacts and advises against commercial or real-world deployment without further testing.
• The original TTS code was removed from the repository in September 2025 after misuse; VibeVoice-TTS is listed as "Disabled" for quick try, though weights remain on Hugging Face.
• Open weights, not open training code. Community discussion points out that training code is not released, so this is open-weight rather than fully open-source in the traditional sense (HN thread).
• Output quality varies. Users in the same thread report hallucination, heavy inference, weaker multilingual performance, and noisy training data for the 7B TTS release (comment, reply). These are individual impressions, not benchmarks.
• Misuse risk is explicit: the README warns about deepfakes and disinformation, and asks users to disclose AI-generated audio and comply with local law.

HotRepo's take

VibeVoice reached #13 on the GitHub all-language daily ranking on 2026-11-05, with 55,376 stars and 6,241 forks at that snapshot, after adding 214 stars that day; earlier snapshots in late July 2026 showed gains around 330 per day. The project's broad attention is also visible in an April 2026 Hacker News thread with 386 points and 181 comments — a discussion where skepticism about the "open source" label and about inference cost was as prominent as interest. That combination — high star counts alongside pointed criticism of weight openness and runtime performance — makes VibeVoice a reasonable subject for benchmarking and research, but a project whose production readiness should be judged on your own audio and latency requirements rather than on popularity alone.