joeblau/subvox

What it does

Subvox is a macOS prototype exploring whether headphone head-motion data from CMHeadphoneMotionManager carries enough information to classify silently mouthed words. It uses no language model: recognition is a small-vocabulary template matcher, paired with a statistical test that reports whether the motion signal separates your words better than chance.

The app has three screens. Sensors plots acceleration, rotation rate and attitude and records experiment sessions. Train captures word templates by holding Space while mouthing a word, then runs leave-one-out classification plus a 500× label-permutation test to report accuracy, a p-value and a confusion matrix. Decode matches live segments against templates with DTW and shows a parallel mic transcript for comparison.

The decoding pipeline is: segment the motion stream (CaptureController), extract per-frame features (MotionFeatures), classify with 1-nearest-neighbour DTW under a Sakoe–Chiba band (DTW, TemplateClassifier), and test significance (Evaluation). Segments are rejected when the best distance exceeds max distance or the margin over the runner-up is below min confidence. See the decoding description in the README.

Build and run

Prerequisites: macOS 14 or later, and AirPods Pro or another headphone supporting head tracking, connected as the Mac's audio output. Install XcodeGen once, generate the project, then open and run it:

brew install xcodegen
xcodegen generate
open Subvox.xcodeproj   # ⌘R

A command-line build and test path is also documented:

xcodebuild -project Subvox.xcodeproj -scheme Subvox -derivedDataPath build -allowProvisioningUpdates build
open build/Build/Products/Debug/Subvox.app
xcodebuild -project Subvox.xcodeproj -scheme Subvox -derivedDataPath build test

On first launch, grant Motion permission. Microphone and speech prompts appear only when you start the mic. The build instructions are the authoritative source for this workflow.

Experiment protocol

Record sessions with Record Session in the toolbar or on the Sensors screen. Data is written to ~/Documents/Subvox/Sessions/<timestamp>-<condition>/ as motion.csv, audio.caf (if the mic is running), events.csv and meta.json; the last includes the audio start offset used to align audio with motion. The README suggests running the same 5-word script under Normal speech, Whisper and Silent mouthing, 10 reps each, then comparing Evaluate results across feature sets. Training data is stored at ~/Library/Application Support/Subvox/dataset.json (protocol).

Known limits

Headphone motion is processed and low-rate (roughly 25 Hz on AirPods), and the speech-detecting accelerometer and inward mic are not exposed, so the signal reflects jaw and head movement rather than vocal vibration. Templates are user- and session-specific; re-seating the AirPods can shift the signal. Selecting AirPods as the mic input switches them to the Bluetooth headset (HFP) profile (known limits).

HotRepo's take

On the HotRepo new-project list for 2026-11-01, Subvox was recorded at 442 stars and 69 forks, with 183 stars gained over a window starting 2026-10-31 20:45 UTC and observed through 2026-11-01 09:45 UTC. Hourly sampling over that window rose from 259 to 442 stars. No overall GitHub daily rank is recorded. What stands out technically is the framing: this is not a speech-to-text product but a falsifiable experiment, with a permutation test and explicit feature toggles so the claim can be checked rather than assumed. The stated limits on signal rate and session-specific templates are what any reader should weigh before expecting usable silent dictation.