Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering

Noise Out, Bias In. Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering. By Sarim Hashmi and five others, from the Mohamed bin Zayed University of Artificial Intelligence. Published on the fifth of October. Its code repository is worked out: about seventy-five source files, with a readme.

Masked diffusion language models, dLLMs, generate text by iteratively denoising masked positions, re-predicting each token several times before it is committed. That differs from an autoregressive decoder in a way that matters for security. An autoregressive decoder exposes the distribution over an answer once, at the step that commits it. A dLLM exposes it at every denoising step before commitment. One gives an observer a single glimpse of the answer's distribution, at the moment it becomes final. The other gives many glimpses, spread across the whole time the answer is still undecided.

The authors show that an adversary can exploit this. Because an answer stays open to revision over many denoising steps, an adversary with access to internal activations can watch how likely the model is to produce a chosen answer, and adjust the intervention accordingly. The model is, in effect, reporting its own progress to whoever is steering it.

The attack built on that observation is called targeted bias injection. It steers a frozen dLLM toward a demographic answer that the adversary selects. The weights are never changed. What changes is how the model is pushed while it generates, and how that push responds to what the model is doing.

The push comes from a steering vector, and what is new is how its strength is set. A simple proportional-integral controller, PI for short, tracks the probability of the target answer during denoising and adapts the strength of the steering vector on the fly. Rather than fixing the strength in advance, the attack treats the denoising process as a loop to be controlled, with the target probability as its feedback signal. The controller is described as simple, and that matters for the threat: what makes the attack possible is the access the denoising process gives, not elaborate machinery on top of it.

The main test uses ambiguous BBQ questions, where the correct answer is abstention. There the attack raises the preference of LLaDA-8B-Instruct for the targeted group from one point eight to sixteen point seven percentage points. That is more than three times the strongest fixed-strength steering baseline. On SocialStigmaQA, it raises the selection of stigmatizing answers from seventeen point six percent to fifty-eight point one percent.

Fitted to other demographic targets, the same attack shifts answers by up to thirty-seven percentage points. Each attack takes about forty minutes on one GPU, which puts it within reach of anyone who has the activation access to begin with.

On the primary target, the authors isolate the feedback itself. Constant steering, at the same average strength over the token-committing steps, produces a far smaller shift while corrupting nearly three times as many outputs. A constant strength set separately for each example still falls well short. Feedback is what makes the attack work, not the steering vector alone. Pushing harder without watching does more damage and achieves less; pushing in response to the model's own trajectory is both stronger and cleaner.

The lesson reaches past this one attack. The denoising trajectory is, in the authors' framing, a new control channel in dLLMs, a place where an answer can be shaped while it is still forming. Because the attack lives in how the model is run rather than in its weights, an audit of the frozen model would not see it. Hence their call for bias audits that examine the serving stack rather than the frozen model alone.