jamesondh/chimera-vocal-synth

Project overview

Chimera is a physical-modeling voice synthesizer that runs in the browser. Instead of playing back recorded samples, it simulates an airway — lungs, vocal folds or syrinx, a branched vocal tract, lips and nose — so every sound is produced by the model. The engine is written in Rust, compiled to WebAssembly for an AudioWorklet, and the same code also runs natively for tests and the chimera CLI.

The browser interface exposes several modes:

• Play: drag the tongue, lips and jaw in a live cutaway of the head; use the vowel pad, encoders, keyboard or Web MIDI.
• Speak: English text (CMUdict with a letter-to-sound fallback) or IPA becomes gestures and prosody, with the gestural score and a consonant chart on screen.
• Sing: a piano roll with lyrics per note, with consonants placed before the beat.
• Beatbox: pads and a step sequencer where each drum is an articulatory gesture.
• Bestiary: songbird (two-voice syrinx), seal, alligator and elephant models.
• Duet (?show=<id>): two 3D heads replay a pre-rendered song.

Male, female and child bodies blend continuously. On-screen animation — airway pressure, airflow, vocal folds, spectra and formants — is driven by live engine state. The project is aimed at people interested in voice synthesis, articulatory modeling and interactive audio, and at developers who want to test the engine through the CLI or inspect the model in the architecture docs.

Quick start

Prerequisites: Rust with the wasm32-unknown-unknown target, and Bun.

bun install
bun run dev        # builds the wasm, then Vite on 127.0.0.1:5317

The AudioWorklet requires a secure context, so open the app on localhost or over HTTPS (for example tailscale serve). To run the test suite:

bun run test       # wasm build, cargo test --workspace, bun test tests

Offline renders and measurement are available through the CLI:

cargo run --release -p chimera-cli -- render speech|text|song|notes|beatbox|creature|vowels …
cargo run --release -p chimera-cli -- diagnose       # nasal leakage, formants, bandwidths, pitch
cargo run --release -p chimera-cli -- compare csd    # against a real singer (docs/csd-comparison.md)
bun tools/spectrogram.ts in.wav out.png

The hosted version is available at chimera-vocal-synth.vercel.app.

Usage constraints

• The browser AudioWorklet needs a secure context; plain HTTP on a non-localhost address will not work.
• Building the web app requires both the Rust wasm target and Bun.
• The CLI render subcommands accept several content modes; the argument lists are shown as ellipses in the README, so check the CLI itself or the docs for exact flags before running a full render.
• The project is MIT licensed, but bundled CMUdict data and fonts keep their own licenses.

Documentation

The architecture document covers the acoustic model (FDTD airway, sources, anatomy, gestures). The real-time and determinism contract explains the engine boundary. Voice quality is assessed against the Children's Song Dataset in docs/csd-comparison.md, and the hill-climb log records tuning steps. Agent-facing notes live in AGENTS.md.

HotRepo's take

The repository appeared in HotRepo's Rust new-project board on 2026-11-08, with 136 stars and 33 stars gained in a recorded window of about 90 minutes; its stars rose from 103 to 136 in the collected samples that morning. The README and docs describe an actual simulation engine, a browser UI and a native CLI, and the stated model is testable without the web build. The scope is ambitious — speech, singing, beatbox and animal models in one engine — so the practical question for adopters is how mature each mode is rather than whether the project exists.