NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

NOAH. Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting. By Tobias Susetzky and seven others, from TUM University Hospital, the Technical University of Munich, and three other institutions. Published on the eighth of September. No code repository is linked to it.

The digitisation of healthcare has produced vast patient records: longitudinal, multimodal, and spanning a lifetime. Using them fully, to represent where a patient stands and to predict where their state is heading, remains a critical challenge. Current AI models often struggle with the complex, irregular timing of real patient data and with its inherent randomness. Existing approaches to longitudinal records have characteristic limits. Most are discriminative. Many cover only a few modalities, or are tied to closed categorical vocabularies. Some treat time only as a monotonic inductive bias. And many are limited in how well they forecast future patient states.

NOAH is a generative transformer that is time-aware and task-agnostic, meaning it is not built around one prediction task, and designed to represent and forecast the full multimodal patient journey. Two features carry that aim. A new bidirectional time integration handles the continuous evolution of patient states. A variational latent space captures the stochasticity of clinical trajectories, the fact that the same history can lead to different futures.

It is built from over five hundred and fifty-nine million clinical events, drawn from four hundred and thirty-one thousand hospital visits by two hundred and ninety-nine thousand patients, across the MIMIC dataset family. NOAH processes many kinds of data natively: medical images, time series and numeric signals, categorical events, and both structured and unstructured clinical records. Images, signals, coded events and written notes all enter the same model, which is what lets it follow the patient journey as a whole rather than one kind of record at a time.

Being generative gives it a range of uses from one model. It can forecast autoregressively, with optional control over timing. It can classify zero- shot. And it can simulate counterfactual interventions, asking what the trajectory would look like under a different course of events. The authors describe it as the first truly holistic generative model in its field.

The representations NOAH produces of a patient's state are highly informative and predictive. Probed for clinical outcomes, they perform strongly, including across fifteen ICD chapters and twenty-nine comorbidities. They also perform strongly in time-to-event prediction, where the question is not only whether something happens but when. Those probes test whether the learned representation already contains the information the clinical questions need, without building a separate model for each question.

Each limitation listed at the start has a counterpart in the design. Discriminative models give way to a generative one, few modalities to many, closed vocabularies to open handling of records, fixed treatments of time to a bidirectional one, and weak forecasting to a model built to forecast. The authors present it as a versatile, scalable foundation for predictive systems in personalised clinical care and digital medicine.