NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

NitroFusion2. Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers. By Dar-Yen Chen and one other, from the University of Surrey and Turing Elite Research Labs. Published on the third of November.

Adversarial diffusion distillation depends on a discriminator, and that discriminator has conventionally judged samples by their image features. The authors question that default for one family of models in particular: multimodal diffusion transformers, or MMDiTs. In these models the text stream and the image interact through joint attention, so the text stream does not stay purely about the prompt. It becomes dependent on the image. The conventional choice leaves that image-aware text representation unused.

Text-Stream Adversarial Distillation, or TSAD, uses it. The intermediate states of the text tokens become a new representation for the adversarial game. What makes them useful is a dual property. The text stream has already read the image, yet it is still tied to the text condition. Change the condition, and you change what information the stream extracts from the image. The text stream, in other words, is a representation of the image that can be pointed at different things, simply by changing what it is conditioned on.

That property is what Aspect-Cued Discrimination, or ACD, builds on. ACD conditions the discriminator on different aspects of the sample, such as typography, objects, or style. By choosing the aspect, training can steer which properties of the image the adversarial signal emphasizes. Both pieces are then integrated on top of the expert-discriminator design of the original NitroFusion, and the combination is NitroFusion2.

Few-step distillation has conventionally operated at four to eight steps. NitroFusion2 brings that down to one or two steps, at comparable quality. At a single step it produces sharp rendered text and rich fine-grained detail, which the authors argue makes one-step generation practical rather than merely a stress test for distillation. At two steps it stays competitive across the benchmarks evaluated. Put side by side, the step counts tell the story: where the conventional regime needed four to eight passes, one pass now gives usable images, and two give competitive ones.

The shift is in where the discriminator looks. Image features were the habit. In an MMDiT, the text stream turns out to carry a view of the image that can be redirected just by changing the condition, and the authors present it as a powerful and steerable representation for adversarial distillation.