Dettato: Tiny open-source model turns music into MIDI and beats much bigger rivals

Dettato turns music recordings into MIDI notes. With just 20 million parameters, it beats MuScriptor, a model 65 times its size, in benchmarks and narrowly edges out a commercial model too. It was trained without any copyrighted recordings, and you're free to use its output commercially.

Model releases Try it yourself Nov 17, 2026 6 min read Source: Show HN (neue Projekte)
Illustrative image, AI-generated

What it's about

The developer behind kobimusic has introduced a new AI model on Hacker News: Dettato. It listens to a music recording and writes down which notes are played, when, for how long and at what pitch. The output is a MIDI file. MIDI is a digital note format. It doesn't store sound, only instructions like "key C4, medium velocity, half a second long". You can load these files into any music software, play them back with other instruments, edit them or process them further.

The technical term for this task is Automatic Music Transcription (AMT). According to the developer, Dettato is open source, and you can use its output commercially. The project page with details is at kobi.music/research/dettato. The developer says Dettato is currently the best model of its kind, and it's tiny: just 20 million parameters. MuScriptor, which the developer calls the best open model until now, has 1.3 billion, so it's roughly 65 times larger. For comparison, even small language models these days usually have several billion parameters.

The benchmarks

The developer compares Dettato with three other systems. They're scored on four subtasks, and the average of those gives an overall score:

  • Onsets: Is the start of a note detected correctly?
  • Frames: For each short slice of time, are the right notes sounding?
  • Offsets: Is the end of a note detected correctly? This is considered especially hard.
  • Drums: How well are the drums picked up?
System Overall Onsets Frames Offsets Drums Parameters
Dettato 60.8 70.8 79.1 40.9 52.5 20M
Mirelo A2M v1.1 Pro (commercial) 56.1 67.1 75.7 40.1 41.4 not disclosed
MuScriptor 47.4 57.0 71.2 31.4 30.1 1.3B
YourMT3+ 45.7 58.2 63.9 30.5 30.0 46M

The biggest lead is on drums: 52.5 versus 41.4 for the commercial Mirelo model. On offsets, Dettato and Mirelo are almost neck and neck.

Broken down by test dataset, the picture is more mixed. The scores here are F1 scores, a measure from 0 to 100 that combines precision and recall, averaged over onset, frame and offset:

Dataset Dettato Mirelo A2M v1.1 Pro MuScriptor 1.3B YourMT3+
Bach10 91.3 – 54.7 57.9
Dagstuhl ChoirSet 60.8 – 35.5 28.0
PHENICX-Anechoic 64.8 60.9 54.4 44.8
RWC Popular 54.0 – 44.3 36.1
RWC Classical 64.4 65.8 58.4 62.4
RWC Genre 48.7 – 42.6 35.6
RWC Jazz 55.7 – 52.0 47.1
RWC Royalty-Free 61.6 56.2 46.8 45.4
Slakh2100 (synthetic) 68.4 – 51.1 –

On classical music from the RWC collection, Mirelo comes out slightly ahead. Mirelo only has scores for three of the nine datasets at all. So the comparison with the commercial model relies on numbers its maker published itself.

Trained without copyrighted recordings

The training data is unusual. According to the developer, not a single copyrighted recording was used. Instead, he used:

  • synthetic instrument recordings, meaning computer-generated music,
  • Creative Commons solo instruments from the URMP dataset, which were mixed in,
  • about 20 soundfonts, collections of recorded instrument sounds that let a computer make MIDI notes audible. He "morphed" these into one another with a home-grown technique called "Soundfont Babies", blending them into new timbres.

Behind this is kobimusic's goal of helping musicians rather than replacing them. For the developer, that includes respecting copyright. His takeaway:

The constraint of not using copyrighted recordings doesn't hold you back. It drives innovation, and you probably end up with the better model.

Why such a small model can be this good

Work on Dettato started about ten months ago. In April, the developer threw out everything he had. He had been treating transcription as a data problem, but the real bottleneck, he says, was the architecture, the blueprint of the model.

He draws a parallel to his first model for OMR (Optical Music Recognition), which reads printed sheet music from images. Both fields were first tackled with classic algorithms, and later with huge language-model building blocks. Large models like these look at the problem "globally", taking in the piece as a whole, and they adapt too closely to synthetic training data. That's called overfitting: the model memorizes the quirks of its training data instead of learning general rules. His approach is to look at the problem locally. For OMR, a detector model does that. For music transcription, it's a pitch-lane model, which roughly speaking splits the music into individual pitch lanes and looks at each one on its own. The post doesn't go into further technical detail.

Limitations the developer names himself

The developer is remarkably upfront about the weaknesses. Benchmarks say little about how well the model works for your use case, he says, so you have to try it yourself. Dettato is weak at separating instruments: when several instruments play at once, it often assigns notes to the wrong one. It also produces beat-aligned performance MIDI, a record of what was actually played. You won't get playable sheet music or clean engraving out of it. If you need sheet music, the developer recommends the commercial services Songscription or Klangio.

What you need to try it

At 20 million parameters, Dettato is extremely small. The weights should take up less than 100 megabytes. You probably won't need a graphics card, and a regular laptop with 8 GB of RAM should be enough. A Mac or Google Colab should work too. One caveat: the post mentions no Hugging Face repository, install commands or exact license. To find out how to download and run the model, head to the project page. Check the license terms there before you use it.

The bigger picture

Dettato fits a trend you can see outside music too: a small, purpose-built specialist model can beat much larger general-purpose approaches. That's handy for musicians, producers and music educators. A model this small runs locally, fast and cheaply. And in an industry full of copyright fights, legally clean training data plus free commercial use of the output is a real selling point.

A few caveats: all the numbers come from the developer himself. There's no independent verification yet, and the comparison with Mirelo rests on just three datasets. The model's poor instrument separation is a noticeable gap for band recordings. And if what you ultimately need is sheet music to play from, Dettato alone won't get you there.

Sources

#audio-to-midi#music#transcription#open source#small model#copyright
Read the original source

Try it yourself: Dettato

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

Dettato listens to a music recording and writes the notes played into a MIDI file: pitch, start and end of each note, aligned to the beat. It also picks up drums, and according to the developer's benchmarks it does so better than any of the models it was compared with. It's especially strong on classical music and choirs, scoring 91.3 on the Bach10 dataset, for example. On pop and mixed genres it lands at around 49 to 54. Overall, it clearly beats MuScriptor, a model 65 times larger, and narrowly beats the commercial Mirelo A2M. It struggles to keep several simultaneous instruments apart. And it doesn't deliver print-ready sheet music, only performance MIDI. According to the developer, you can use the output commercially.

Who is it worth it for?

Dettato is worth a look for musicians, producers, remixers and music teachers who want to pull melodies, accompaniments or drum patterns from recordings into their DAW, and for developers looking for a small transcription model that runs locally. If you need finished, playable sheet music or want band recordings cleanly split by instrument, the developer says you're better off with services like Songscription or Klangio.

Ideas: What you can do with it

Rebuild a drum groove

Take a drum loop from one of your own recordings, have Dettato turn it into MIDI and drag the file into your DAW. There you can play the pattern with any drum samples you like and tweak it.

Capture a piano improvisation

Record a spontaneous improvisation on an acoustic piano with your phone. Dettato turns it into MIDI that you can edit in your DAW or play back with a different sound, like Rhodes or strings.

Practice material for music lessons

As a teacher, have a choral setting or a chamber piece transcribed so you can mute individual parts in the DAW or slow them down. Students can then practice their own part in isolation.

Lift a melody for a remix

Transcribe a guitar or vocal line you recorded yourself and have a synth play it. Because the output can be used commercially, you can release the result too, as long as the original recording is yours.

Build your own music app

As a developer, build the small model into an app that converts recordings to MIDI locally, with no cloud costs. Thanks to its tiny size, that's doable even on weaker hardware.

Download0.1 GB (about 1 min at 100 Mbit/s)
Licenseapache-2.0
TypeAudio model
Hugging Facekobimusic/dettato

What your computer needs

Graphics card (NVIDIA) at least 4 GB VRAMThe maker measured this on an RTX 5060 with 8 GB. A 4-minute song takes about 15 seconds there, a 6-minute song about 25 seconds, and both stay under 4 GB of VRAM (video memory). Beyond about 250 seconds of music, the program offloads intermediate data to regular RAM (around 10 MB per second of audio) so VRAM usage doesn't keep growing. It also runs without a graphics card, but slowly: a laptop CPU took 87 seconds for 15 seconds of music, including model loading.
Mac (Apple Silicon M1–M5) from 8 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it
Storage approx. 2.1 GB freeModel download plus programs

The developer only mentions one variant, with 20 million parameters. Given how small it is, you don't need quantization or any other shrinking. The model should run on virtually any current computer, even without a graphics card. You'll find the download and setup instructions on the project page kobi.music/research/dettato.

  • At 20 million parameters, the model is very small. The weights should come in under 100 MB (an estimate, not an official figure).
  • You probably won't need a graphics card. Running on the CPU should be fine for individual songs.
  • About 8 GB of RAM should be enough. Very long recordings may need more.
  • Apple Silicon Macs and Google Colab should work too. The post gives no official information on supported platforms.
How do I find out what my computer has?
  • Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
  • Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
  • Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

This model runs with its developers' own program (kobimusic/dettato on GitHub). The commands below come from the Model Card and the developers' instructions.

Windows 10 / 11

Windows 10 / 11: not officially tested. The source doesn't mention Windows. As a Python package, though, dettato should run. With an NVIDIA card, you need to install a PyTorch build with CUDA, because the standard Windows package only uses the CPU. The PyTorch website gives you the right command. If all else fails, WSL2 with the Linux instructions works too.

  1. Install the prerequisites. Install Python 3.10 or newer and tick "Add Python to PATH" during setup. Also install Git for Windows. If you have an NVIDIA card, you need the latest driver. For m4a, aac and some mp3 files, ffmpeg must be installed and on your PATH, so you can call it from anywhere. Check everything in PowerShell:
    python --version
    git --version
    nvidia-smi
  2. Create a working folder. Open PowerShell and create a dedicated folder.
    mkdir dettato-test
    cd dettato-test
  3. Create a virtual environment. The venv keeps the packages separate from the rest of your system. If Windows says scripts are blocked, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned once first. If it worked, you'll see (.venv) at the start of the line.
    python -m venv .venv
    .venv\Scripts\Activate.ps1
  4. Install PyTorch. dettato needs PyTorch 2.2 or newer. The simple command installs a CPU-only build. For your NVIDIA card, grab the install command for "Windows, Pip, CUDA" from the PyTorch website and run that instead.
    python -m pip install --upgrade pip
    pip install torch
  5. Install dettato. This downloads the program from GitHub and installs it. Afterwards you'll have the dettato command.
    pip install git+https://github.com/kobimusic/dettato
  6. Get a music file ready. Copy a recording into the dettato-test folder as song.mp3. You can find free recordings on Wikimedia Commons, such as Musopen's recording of Beethoven's 5th Symphony. Drums are only detected in recordings with a sample rate of at least 40 kHz.
  7. Run your first transcription. The first time, dettato automatically downloads the models from Hugging Face. Without a graphics card it takes noticeably longer, so try a short piece first.
    dettato song.mp3 -o out/
  8. Check the result. The out folder contains song.mid and song.json. Open the MIDI file in a music program like MuseScore or your DAW.
    dir out
  9. Result. The out folder contains song.mid (MIDI with one track per instrument plus drums) and song.json (info on tracks, note counts and tuning).

macOS

macOS: not officially tested. The source doesn't mention macOS. It only names CUDA (NVIDIA) or the CPU; Apple's GPU backend (MPS) doesn't come up. So on a Mac it will most likely run on the CPU only, which is slow. Stick to short pieces.

  1. Install the prerequisites. Install the Xcode Command Line Tools, which also give you Git. You also need Python 3.10 or newer, for example from python.org or via Homebrew. For m4a, aac and some mp3 files you need ffmpeg, which you can get via Homebrew.
    xcode-select --install
    python3 --version
    git --version
  2. Create a working folder. Open Terminal and create a dedicated folder.
    mkdir dettato-test
    cd dettato-test
  3. Create a virtual environment. The venv is a separate space for your Python packages. Afterwards you'll see (.venv) at the start of the line.
    python3 -m venv .venv
    source .venv/bin/activate
  4. Install PyTorch. dettato needs PyTorch 2.2 or newer. On a Mac, the standard package is all you need.
    pip install --upgrade pip
    pip install torch
  5. Install dettato. This downloads the program from GitHub and installs the dettato command.
    pip install git+https://github.com/kobimusic/dettato
  6. Get a music file ready. Put a recording in the folder as song.mp3, such as one of your own songs or a free recording from Wikimedia Commons. Since the Mac only uses the CPU, it's best to cut out a 15- to 30-second clip for your first test. For drums, the file needs a sample rate of at least 40 kHz.
  7. Run your first transcription. --device cpu tells dettato to use the CPU. The first time, it downloads the models. For 15 seconds of music, this can easily take a minute and a half.
    dettato song.mp3 -o out/ --device cpu
  8. Check the result. The out folder contains song.mid and song.json. You can open the MIDI file in GarageBand, Logic or MuseScore, for example.
    ls out/
  9. Result. The out folder contains song.mid (MIDI with one track per instrument plus drums) and song.json (info on tracks, note counts and tuning).

Linux

Linux: not officially tested. The source doesn't name an operating system. It only requires Python 3.10+ and PyTorch 2.2+ and recommends an NVIDIA graphics card with CUDA. On Linux with an NVIDIA card, this should be the easiest route. It also runs without a graphics card, just slowly.

  1. Check the prerequisites. You need Python 3.10 or newer and Git, a tool for downloading code. If you have an NVIDIA graphics card, install the latest NVIDIA driver. For m4a, aac and some mp3 files you'll also need ffmpeg. These commands show you the installed versions.
    python3 --version
    git --version
    nvidia-smi
    ffmpeg -version
  2. Create a working folder. Create a dedicated folder for dettato and switch into it. Your results will end up here too.
    mkdir dettato-test
    cd dettato-test
  3. Create a virtual environment. A virtual environment (venv) is a separate space for Python packages, so nothing gets tangled up with your system. If it worked, you'll see (.venv) at the start of the prompt.
    python3 -m venv .venv
    source .venv/bin/activate
  4. Install PyTorch. PyTorch is the compute library the model runs on. You need version 2.2 or newer. On Linux, the standard package usually already includes support for NVIDIA cards.
    pip install --upgrade pip
    pip install torch
  5. Install dettato. This command downloads the program straight from GitHub and installs it. Afterwards you'll have the new dettato command.
    pip install git+https://github.com/kobimusic/dettato
  6. Get a music file ready. Put a recording in the folder as song.mp3, such as one of your own songs. You'll find free recordings on Wikimedia Commons, for example Musopen's recording of Beethoven's 5th Symphony. Important: drums are only detected if the file has a sample rate of at least 40 kHz. Regular 44.1 kHz mp3s are fine. WAV files are the safest bet without ffmpeg.
  7. Run your first transcription. On first launch, dettato downloads its four small models from Hugging Face. They're only a few megabytes. If there's a graphics card, the program uses it automatically.
    dettato song.mp3 -o out/
  8. Check the result. The out folder now contains song.mid and song.json. You can open the MIDI file in any music program, such as MuseScore, LMMS or your DAW. The notes aren't quantized to a beat grid; they follow the exact timing of the recording.
    ls out/
  9. Result. The out folder contains song.mid (MIDI with one track per instrument plus drums) and song.json (info on tracks, note counts and tuning).

What to do after your first result

  • You can pass multiple files and whole folders at once, for example: dettato song.mp3 other.wav folder/ -o out/. Subfolders are searched too, and their structure is preserved in out/.
  • It works from Python too: import dettato; result = dettato.transcribe("song.mp3"); result.save_midi("song.mid"). result.tracks shows you every track with its instrument name and note count.
  • Don't expect perfect instrument assignment. With jazz, for example, about a quarter of the saxophone notes end up in the piano track. The notes aren't quantized to a beat grid. For playable sheet music, the developer recommends Songscription or Klangio.
  • Offline: download the four repos kobimusic/dettato, dettato-evidence, dettato-grouper and dettato-drums into one folder with hf download. Then point the DETTATO_WEIGHTS environment variable at that folder.

Try it

A short solo piano recording (30 seconds, WAV) to see how accurately pitches and note lengths are detected.
A Bach chorale recording or a string quartet, the kind of music where Dettato does best in the benchmarks.
A drum loop from your own rehearsal room, so you can bring the pattern into your DAW as MIDI and play it with other drum sounds.
A guitar melody you recorded yourself, whose MIDI you then have a synth play in your DAW.
A full pop song with a band, to test the limits, especially how well the model keeps the individual instruments apart.

Tips for better results

  • Start with clean recordings of just a few instruments. The sparser the sound, the better the model can tell the notes apart.
  • If you're after a single instrument from a band song, split the tracks first with a stem-splitter tool and transcribe them one by one. That works around the weakness in instrument separation.
  • Load the MIDI into your DAW and focus your corrections on note endings (offsets). That's where detection is weakest, at around 41 points.
  • Make deliberate use of drum detection. According to the benchmark, that's where Dettato leads the competition by the widest margin.

If something goes wrong

The announcement post mentions no Hugging Face repository or install commands. You'll find current download and setup instructions at kobi.music/research/dettato.

The post doesn't name the model's exact license, only that it's open source and that the output can be used commercially. Check the license before you use it.

When several instruments play at once, notes often get assigned to the wrong instrument. That's a known weakness the developer points out himself.

The output is performance MIDI. If you open it in a notation program, you won't get clean sheet music, but rather messy notes with odd lengths.

Scores for pop and genre mixes are much lower than for classical. Expect to do some cleanup on recordings like these.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.