LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows

LynnReal-Omni. Native multi-modal Video Generation for Agentic Visual Workflows. By Xiaofeng Mao and five others, from LynnReal AI. Published on the fourteenth of September. Its code repository is worked out: about one hundred and forty source files, with a readme and a licence.

Video diffusion models are stochastic and hard to control. Getting precise content often requires repeated sampling with no guarantee of success, and long-horizon scenes drift in appearance, in interactions, and in temporal coherence. Agentic visual creation provides stable control, through explicit references, editable 3D scenes, or executable game states, but on its own it does not guarantee high fidelity for objects or characters. Each side has what the other lacks, and combining the two could enable generation that is both stable and high in quality. Diffusion brings the fidelity but not the control. The agentic tools bring the control but not the fidelity.

LynnReal-Omni is built to realize that combination. It is a native multimodal video generation framework on a thirty-two billion parameter shared multimodal diffusion transformer. One model unifies text-to-video, image-conditioned generation, reference-guided generation, structural control, editing, restoration of degraded video, and long-video generation. Seven kinds of task share one set of weights.

It accepts heterogeneous visual inputs, including appearance references, editable 3D renders, and game recordings, which lets agents compose visual conditions within a unified model. The agent supplies the scene; the model renders it. For real-time rendering, the authors also train a dedicated Flash model, a twenty-seven billion parameter shared multimodal diffusion transformer. LynnReal-Omni-Flash further reduces inference cost through model and decoding acceleration, including a lightweight VAE decoder. In practice, an agent can build a scene in 3D, record a game, or gather reference images, and pass any of these to the same model as conditions.

Behind both models is a systematic data pipeline covering video cleaning, subject association, multimodal annotation, and construction of aligned controls, which yields a curated corpus of multi-shot audiovisual segments. For evaluation, the authors introduce MSAVP, a design of one hundred prompts and twenty metrics that separates instruction following, generating plausibility, visual quality, temporal behavior, and audio coordination. The evaluation design is meant to keep these qualities apart, so that a model strong in visual quality but weak in audio coordination, say, shows up as such.

The speed figures are given for one H100. Warm generation and decoding of a twenty-two frame video at five hundred forty p takes eight hundred forty-three milliseconds with LynnReal-Omni, and three hundred seventy-seven milliseconds with Flash. The Flash model is more than twice as fast on this measurement, which is what makes it the option for real-time rendering.

The authors present these results as a foundation for real-time streaming video generation. The division of labour is the design: agents provide the references, scenes and game states that make control stable, and one unified, controllable model provides the fidelity, fast enough in its Flash form for rendering in real time. Stability from the agent, fidelity from the model, and speed from the Flash variant: those are the three pieces the abstract assembles.