k2-fsa/sherpa-onnx

sherpa-onnx is an Apache-2.0 C++ toolkit for on-device speech processing built on next-gen Kaldi and onnxruntime. It covers streaming and non-streaming speech-to-text, text-to-speech, speaker diarization/identification/verification, speech enhancement, source separation, VAD, keyword spotting and related audio tasks without a network connection README. That makes it relevant to anyone embedding voice features in an app or appliance rather than calling a cloud speech API.

What changes in a workflow

Instead of sending audio to a hosted service, an application loads a local ONNX model and runs inference on the user's device. The repository documents that local execution explicitly and lists deployment targets including x86, ARM32/ARM64, RISC-V, Android, WearOS, iOS, HarmonyOS, NodeJS, WebAssembly, Raspberry Pi and several NPUs (RKNN, QNN, Ascend, Axera, OpenVINO) README. Bindings exist for 12 languages, from C++ and Python to Java, Kotlin, Swift, C#, Go, Rust, Dart and Pascal, plus a WebAssembly path.

A concrete example is the Voltip dictation app: it holds a hotkey, records speech, shows a live partial transcript from a streaming Zipformer model, uses Silero VAD to trim silence and split long recordings, then inserts text into the focused application using SenseVoice or Paraformer Voltip. The same building blocks appear in subtitling tools such as SmartSub SmartSub and captioning apps for hearing-impaired users Loud Caption.

Two mechanisms worth knowing

• Streaming versus offline models. Streaming transducers/paraformer/CTC models produce partial results while audio arrives; non-streaming models transcribe a finished utterance in one pass. The README indexes model lists at different pages for each, so model choice follows the interaction pattern README.
• Pipeline composition. VAD, keyword spotting, speaker embeddings and ASR can be combined in one process; the bot-reborn project runs VAD and wake-word detection in a single browser WebAssembly build so only real speech reaches speech-to-text bot-reborn.

Trying it and prerequisites

Several Hugging Face Spaces and prebuilt Android APKs let you test recognition, synthesis, diarization and subtitle generation in a browser or on a phone without compiling anything README. Building an application requires the SDK or language package for your target plus the matching model files; the README notes, for instance, that iOS Flutter demos must be built from source even though other platforms have prebuilt apps.

Compatibility is documented per platform, architecture and framework table, so the practical first step is checking that matrix for your target rather than assuming one build fits all devices. The README does not provide accuracy benchmarks or latency figures, so on-device resource use must be measured against your chosen model and hardware.