Simon Edwardsson's 14-experiment retrospective on building piano autocomplete: connect a MIDI keyboard to a phone, play a few notes, let a 125M-parameter transformer continue the performance in real time (~108 notes/sec on iPhone 15; free app RollTab). Key engineering lessons: the naive NOTE_ON pitch*velocity tokenization explodes the vocabulary to 16,512 tokens, so representation choice dominates; aggressive cleaning of training MIDI (keeping piano-like material, removing/reducing other tracks) mattered more than scale; DPO post-training produced the clearest subjective jump in continuation quality. Includes audio samples (Pokemon Pallet Town, FF6 Terra's Theme, Fur Elise) and demo video.
Source: https://simedw.com/2026/08/20/midi-autocomplete/