DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription

DuET. Note-Frame Dual Encoding for Piano Performance-to-Score Transcription. By Wei Zeng and one other, from the National University of Singapore. Published on the fifteenth of October. Its code repository is public and real: about twenty source files.

Turning an expressive piano performance into a written score means reading each note's timing against a metrical structure that sits underneath it. The difficulty is a mismatch in where the information lives. Score attributes belong to notes. Beats do not: a beat can fall where no note begins at all. A model that looks only at notes has nowhere to put a beat that lands between them, and a model that tracks only beats has no direct handle on the notes it has to notate.

DuET, a Dual-Encoder Transformer, learns both views at once. One task is note-level transcription, from performance MIDI to score. The other is frame-level tracking of beats and downbeats. Rather than choosing between notes and frames, the model keeps an encoder for each and lets the two inform one another.

Bidirectional cross-attention connects the note encoder and the frame encoder. An autoregressive decoder then alternates between their two representations as it predicts structured score attributes. Two positional encodings complete the design. TimeRoPE establishes a shared reference in performance time, so both encoders sit on the same clock. Time-Pitch Rotary Positional Encoding, called TPRoPE, explicitly encodes the relative relationships between events in time and in pitch.

Evaluation uses two datasets. ASAP is a classical piano dataset. Gangqinpu is curated by the authors, spans multiple genres, and focuses on popular piano music. On both, the improvements show up in score timing and in notated duration. Against the strongest baseline on each dataset, DuET lowers MUSTER mean error on Gangqinpu from six point five six percent to three point six four percent. On ASAP the change is smaller, from eleven point three zero percent to eleven point two two percent. Ablations back three ingredients: frame-level context, time-based positional encoding, and masking of the decoder's history.

Code, weights and demos are released. Read as a whole, the argument is about representation. Notes carry the score, frames carry the beat, and transcription improves when one model holds both and keeps them on a shared timeline. The size of that improvement differs sharply between the two datasets: large on the popular music set, slight on the classical one.