YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction

YANAMI. A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction. By Shengyu Wang and twelve others, from the Hong Kong University of Science and Technology, Independent Researcher, and eight other institutions. Published on the third of November.

YANAMI is a large-scale multimodal dialogue dataset and benchmark for stateful interaction, and the authors describe it as the first of its kind. It is aimed at Unified Multimodal Models, or UMMs, which bring visual understanding, generation and editing into a single framework. The gap it fills is this. Existing unified multimodal datasets and benchmarks are dominated by single-turn tasks, and even in recent multi-turn datasets, many turns can be solved independently, without tracking anything across turns. That leaves little supervision or evaluation for stateful interaction, where a model has to understand, maintain, update, and faithfully realize a dialogue state that keeps evolving from turn to turn.

It comes in two parts. YANAMI-400K is the training corpus: four hundred thousand multi-turn multimodal dialogues, with more than two million images and two point six million dialogue turns. The dialogues span four representative scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning. In each, the model must keep tracking, preserving and revising the dialogue state as turns accumulate.

YANAMI-Bench is the evaluation side: four hundred multi-turn dialogues for systematically evaluating stateful interaction. It scores two things jointly, Dialogue State Understanding and Visual Realization. Measuring both lets it characterize the gap between understanding an evolving state and faithfully realizing that state in visual outputs.

The abstract does not say how the dialogues were collected or generated, so this account stops at what it does report: the scale, the four scenarios, and the split into a training corpus and a benchmark.

The corpus is for anyone training unified models to carry state across a conversation. Fine-tuning BAGEL on YANAMI-400K improves image editing by ten point three percent, generation by two percent, and multimodal stateful interaction by thirteen point two three percent, while preserving its general visual understanding. The benchmark is for anyone measuring those models, and its first readings show that existing UMMs still struggle to maintain, update, and visually realize evolving dialogue states.

YANAMI draws a line between a conversation that merely has many turns and one that has memory. Only the second makes a model carry state forward, and that is what both halves are built to train and to test.