adventists-ai/sophon-realtime

What it is

Sophon-Realtime v1 is an open family of full-duplex speech models built on a shared 32B brain. The model listens continuously while you speak, decides at each pause whether you are finished or still talking, and writes replies that include sound marks such as laughter, sighs and emphasis. Three builds differ only in how the reply becomes audio: Tag uses an expressive speech model to perform the tags, Flash uses a lighter and faster speech model, and E2E (preview) generates voice vectors directly inside the 32B model. Weights are published on Hugging Face and ModelScope under Apache-2.0, and the repo also covers an OpenAI GPT-Live compatible gateway and LoRA fine-tuning of the brain.

Quick start

Tag is the recommended build for expressive voice. It needs one 80 GB GPU, Linux, CUDA and Python 3.12. The shortest route installs a package that brings in vLLM-Omni and the Tag plugin, downloads the ~42 GB weights on first run, and serves on port 8000:

pip install sophon-realtime
sophon-realtime serve

The script route isolates the environment and includes a client example:

git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime/tag
bash install.sh
GPU=0 PORT=8000 bash start.sh
python client_example.py --wav my_question.wav --lang en

Flash uses the same pattern in flash/, but downloads the model yourself first, either with modelscope download --model AdventistsAI/Sophon-Realtime-32B-Flash --local_dir ./model or hf download adventists-ai/Sophon-Realtime-32B-Flash --local-dir ./model, then runs start.sh and the client with --voice chelsie. Both builds expose ws://HOST:PORT/v1/realtime for full duplex and POST /v1/chat/completions for audio in, text and audio out. See the Tag build guide and Flash build guide for build-specific details.

Builds and requirements

Tag and Flash each fit on one 80 GB GPU and are served by vLLM-Omni; the project reports about 16 concurrent users per GPU for Tag and 40 for Flash, with first audio around 0.6 s and 0.31 s respectively. E2E needs two 80 GB GPUs and its own engine, with deployment steps in the E2E deploy guide. Prebuilt weights are on Hugging Face for Tag and Flash, with ModelScope mirrors.

Live mode is optional: it is a gateway that speaks the OpenAI GPT-Live protocol in front of a running Tag or Flash engine, so compatible clients only change the URL and key. Setup is described in live/README.md.

Constraints

Serving expects an 80 GB data-center GPU, Linux and CUDA; the quick-start examples assume a single GPU and port 8000. E2E is explicitly a preview and costs twice the GPU memory. The performance figures in the README are self-run benchmarks with stated judges and sample sizes, so they describe the authors' testing setup rather than an independent evaluation. Fine-tuning with LoRA via ms-swift is documented in finetune/README.md, and the code and weights are Apache-2.0, with third-party components listed in each build's NOTICE.

HotRepo's take

The repository appeared in HotRepo's new-project board for Python on 2026-11-10, with 108 stars and 7 stars added over a roughly one-hour window that day; it has no recorded all-language daily rank. That is a very short sampling window, so the numbers show an early burst of attention around launch rather than a trend. The technically notable part is that the project ships both weights and serving code for a full-duplex 32B speech model, and separates expressiveness from latency in two deployment builds instead of asking users to choose one trade-off.