higurashi-ai/Haru

What Haru is

Haru is a low-latency, multilingual, full-duplex spoken dialogue model: it listens and speaks at the same time, like a phone call. The repository contains inference code for Haru-4B, a pretrained base model, with three implementations (PyTorch, MLX for Apple silicon, and a vLLM-Omni plugin) plus a web demo. It is aimed at researchers and developers experimenting with real-time speech interaction rather than end users looking for a finished assistant.

The model processes 133 ms frames, taking one frame of user audio and producing one frame of its own audio plus text; user audio, model audio and model text occupy a single position in the language model. Audio is a 64-dimensional latent per frame generated in one sampling step by a small MLP head, with no discrete audio tokens or separate audio transformer. A text card sets the character and a short reference clip sets the voice.

Run the web demo

Requirements: an NVIDIA GPU with about 12 GB of memory.

pip install -e ./haru
python -m haru.server --hf-repo higurashi-ai/Haru-4B

Then open http://localhost:8998, pick a character, provide a voice reference, and talk. A browser only grants microphone access on localhost or over HTTPS. Haru ships without voice clips, so upload or record a clip of the target voice: one speaker, 3 seconds to 3 minutes, clean speech. Only use voices you have the right to use. For low latency, wired headphones and a wired microphone are recommended; Bluetooth adds 100 ms or more.

To let others try it, add --share after installing the share extra (pip install -e "./haru[share]"); the server prints a public HTTPS link that tunnels through Gradio's share servers. To set a default voice, save a clip as haru/haru/assets/voices/<language>.flac and restart the server.

Use from Python

The DuplexSession API drives the model frame by frame. start() returns frame 0, and each step() with user frame k returns model frame k + 1; each call consumes 3200 samples at 24 kHz and returns 3200 samples.

from haru import DuplexSession, HaruModel
from haru.audio import load_audio, prepare_reference

model = HaruModel.from_pretrained("higurashi-ai/Haru-4B")
voice = prepare_reference(load_audio("voice.wav"))
session = DuplexSession(model, system_prompt=card, reference=voice)

frame = session.start()
while not frame.done:
    frame = session.step(user_audio)

The character card uses a fixed layout built with render_card. For offline batch work, haru-inference user.wav reply.wav --reference voice.wav --system-prompt "$(cat card.txt)" --stereo converts a recording to a reply. On Apple silicon, the MLX implementation includes a native macOS app with build instructions, and haru_vllm can serve many concurrent conversations behind the demo via --backend vllm --vllm-url http://localhost:8000.

Models and constraints

All three implementations read the same Haru-4B checkpoint. Custom Triton kernels speed up the audio codec on NVIDIA GPUs (about 0.7 ms instead of 6 ms per frame, opt-in with DuplexConfig(codec_backend="triton") or --codec-backend triton); MLX uses Metal kernels for the Gated DeltaNet recurrence and codec convolutions. vLLM-Omni serving is intended for GPU servers needing many concurrent low-latency sessions.

Haru-4B is a pretrained base model released for research: it holds a natural conversation but is not a reliable source of facts and does not follow instructions like an assistant. Voice imitation is limited to clips you have the right to use. The demo only exposes the microphone on localhost or HTTPS, and sharing via --share sends traffic through Gradio's servers. Code and weights are Apache-2.0, with third-party components listed in THIRD_PARTY_NOTICES.md.

HotRepo's take

Haru entered HotRepo's new-project list in the Python language board with 101 stars and 3 forks in its first recorded window, from 2026-11-12 to 2026-11-13, taking 101 of those stars inside that single growth window. That is the only snapshot collected so far, so it establishes initial interest but no trend, and the repository has no all-language daily rank on record. What distinguishes it for readers is scope: a full-duplex speech model with working PyTorch, MLX, and vLLM serving paths plus a demo, rather than a paper-only release. The practical trade-off is that the base model's conversational reliability and voice-cloning permission limits should be weighed before building anything user-facing.