kobimusic/dettato
Media & DesignAudio-to-MIDI transcriber that turns recordings into per-instrument note tracks and drum hits for musicians and producers.
What it does
dettato converts an audio recording into a MIDI file with one track per instrument plus a drum track. It chains four small models: a 15.3M-parameter transcriber that detects note onsets, ends, and pitches on the 88 piano keys, a 4.9M evidence repairer that removes wrong notes and fixes timings, a 2.9M grouper that assigns notes to instrument tracks, and a 5.0M drum model. Total size is 28.1M parameters.
The author reports a composite F1 of 59.2 across nine public benchmark sets, versus 45.1 for the 1.3B MuScriptor, and 60.8 against 56.1 for Mirelo A2M v1.1 Pro on the three sets Mirelo publishes. These are the author's own measurements and the four models have not been scored as one chain yet.
Install and run
Requires Python 3.10+ and PyTorch 2.2+. Install from the repository and transcribe a file:
pip install git+https://github.com/kobimusic/dettato
dettato song.mp3 -o out/
This writes out/song.mid and out/song.json (tracks, note counts, tuning estimate, drum model used). Weights download from the Hugging Face Hub on first use; models live in the dettato collection. The CLI accepts multiple files and folders (searched recursively), keeps folder structure under out/, and uses a GPU when present; pass --device cpu to force CPU.
From Python:
import dettato
result = dettato.transcribe("song.mp3")
result.save_midi("song.mid")
for track in result.tracks:
print(track.name, track.program, track.is_drum, len(track.notes))
For batches, instantiate Dettato.from_pretrained("cuda") once and call transcribe per file so models are loaded only once. Each stage can also run separately through dettato.transcriber, dettato.evidence, dettato.grouper, and dettato.drums. To load weights offline, point the DETTATO_WEIGHTS environment variable at a folder containing the four model repos with their config.json and model.safetensors.
Constraints
- Notes are limited to the 88 piano keys, with at most two simultaneous notes per key.
- Instrument naming is approximate; on jazz, roughly a quarter of saxophone or lead-guitar notes can land in the piano track, and synth-heavy and concert-band material gets less reliable names.
- The drum model needs audio recorded at 40 kHz or higher and is skipped for lower-rate sources.
- Output is performance MIDI: note timing follows the recording and is not quantized to a beat. For beat-aligned or playable scores, the author points to Songscription or Klangio.
- A CUDA GPU is recommended; on an RTX 5060 (8 GB) a 4-minute song takes about 15 s, while a 15 s clip took 87 s on a laptop CPU.
- Audio is read with
soundfile; formats it cannot handle (m4a/aac, some mp3s) go throughffmpeg, which must be on the PATH. - Code and weights are Apache-2.0, but some grouper and drum-model training datasets carry non-commercial terms.
HotRepo's take
The project entered the new-project list on 2026-11-20 with 10 stars gained in a one-hour window, reaching 111 stars and 8 forks.
Written by AI from the project's README, updated . The project's own docs are the source of truth.