openai/whisper

What It Is

Whisper is a Transformer sequence-to-sequence model trained on a large and diverse audio dataset. Instead of chaining separate VAD, language-ID, and ASR components, it treats multilingual transcription, speech translation, spoken language identification, and voice activity detection as token-prediction tasks for a single decoder. It is relevant to developers and researchers who need an offline-capable speech recognition baseline, and to anyone building transcription into a Python or command-line workflow.

Six model sizes are published: tiny, base, small, medium, large, and turbo. The first four have English-only .en variants, while large and turbo are multilingual only. The README reports approximate VRAM needs from about 1 GB for tiny/base up to about 10 GB for large, with turbo at about 6 GB; relative speed figures were measured on an A100 and vary with language, speaking rate, and hardware.

Install

Whisper requires Python 3.8–3.11 and a recent PyTorch version, plus the ffmpeg command-line tool. Install the package and ffmpeg on your platform:

pip install -U openai-whisper
# Ubuntu / Debian
sudo apt update && sudo apt install ffmpeg
# macOS
brew install ffmpeg
# Windows (Chocolatey)
choco install ffmpeg

If tiktoken has no wheel for your platform, Rust is needed. Installation failures mentioning No module named 'setuptools_rust' can be addressed with pip install setuptools-rust. To track the repository directly, install from git+https://github.com/openai/whisper.git instead.

Quick Start

Transcribe one or more audio files with the default turbo model:

whisper audio.flac audio.mp3 audio.wav --model turbo

The same operation from Python:

import whisper

model = whisper.load_model("turbo")
result = model.transcribe("audio.mp3")
print(result["text"])

The transcribe() method reads the whole file and processes it in sliding 30-second windows with autoregressive predictions. For language detection and single-window decoding, use load_audio(), pad_or_trim(), log_mel_spectrogram(), detect_language(), and decode() as shown in the README's lower-level example.

Translation and Language Selection

To transcribe non-English speech, pass the source language, for example whisper japanese.wav --language Japanese. To translate speech into English, use a multilingual model other than turbo:

whisper japanese.wav --model medium --language Japanese --task translate

The README states that turbo is not trained for translation and will return the original language even when --task translate is specified. The full list of supported languages is in whisper/tokenizer.py, and whisper --help shows all CLI options.

Constraints

• Large models need substantial GPU memory; CPU-only use is slower and not the README's benchmark scenario.
• Accuracy varies widely by language; the paper's appendix provides WER/CER and translation BLEU breakdowns.
• Silent or music-only segments can be hallucinated as subtitle credits or unrelated phrases. A 2025 discussion documents this in Arabic and Chinese, with commenters attributing it to noisy or unofficial subtitle training data discussion Chinese example Czech example.
• Code and model weights are released under the MIT License.

HotRepo's take

The repository recorded 118 new stars in the Python daily snapshot on 2026-11-12, where it ranked second on that list, showing the project remains actively discovered long after its 2022 release. The HN discussions are more useful for understanding its failure modes than its popularity: the July 2025 thread on silent-audio hallucinations drew hundreds of comments and concrete examples across languages, which is a practical warning for anyone running transcription over long recordings with quiet passages. The v2 and v3 release threads track the model's version history rather than independent benchmarks.