SheetSage2: Coherent Lead-Sheet Transcription with Synthetic Supervision

SheetSage2. Coherent Lead-Sheet Transcription with Synthetic Supervision. By Junyan Jiang and six others, from New York University, MBZUAI, and two other institutions. Published on the fourth of October. No code repository is linked to it, though it does publish its weights.

Turning a recording into a score a person can read takes a coherent grasp of rhythm, harmony, melody, and form. Two obstacles stand in the way. Annotated recordings are scarce. And even when each local prediction is accurate, the sequence they add up to can still be musically inconsistent. A chord can be right in the moment and wrong for the phrase. The first obstacle is about having enough to learn from, and the second is about whether the output hangs together as music once it is learned.

SheetSage2 is a unified transcription framework that answers both obstacles with three ingredients: synthetic data, task-specific structured decoding, and autoregressive distillation.

Scarcity is handled by automatically annotated MIDI, rendered into audio, which gives scalable supervision across music understanding tasks. Consistency is handled by task-specific structured decoders, which integrate complementary musical cues and their dependencies over time to produce coherent scores. Autoregressive distillation then keeps the transcription accuracy without needing task-specific dynamic programming at inference.

Evaluation spans eight benchmark collections. A single SheetSage2-AR model exceeds the listed prior systems on twelve of fifteen benchmark and metric pairs in the authors' evaluation. It improves substantially over SheetSage1, and surpasses task-specific models on several benchmarks. Model weights and inference code are publicly available.

Each part of the recipe answers one problem. Synthetic audio from annotated MIDI supplies the labels that real recordings lack. Structured decoding supplies the coherence that local accuracy alone does not guarantee. And distillation lets one model keep both without the extra machinery at inference time, which is why a single model can be set against systems built for one task each.