Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models. By Xiaoqiang Wang and three others, from the Université de Montréal, Mila, the Quebec AI Institute, and one other institution. Published on the twenty-first of September. No code repository is linked to it.

One way to adapt a language model is to change the computation applied to individual tokens. Quantum circuits offer one such approach. The difficulty is cost. Evaluating wider circuits inside a large model can be computationally demanding, which limits how far the idea can be pushed. The question this work takes on is whether circuits can be made wide enough to matter while staying cheap enough to train inside a real model.

HyperQ adds token-conditioned quantum residual branches to a frozen masked-diffusion language model. In each transformer block, a branch reads a token's hidden state, emits the coordinates of a circuit for that token, executes the circuit, and adds the measured values back through a residual connection. The backbone stays frozen, and only the added branches are trained.

Inside each branch, a lightweight circuit hypernetwork produces the token-specific settings: rotation angles, coupling strengths, and measurement axes, all within a shared sparse circuit structure. So every token gets its own circuit, but all the circuits share one layout.

The cost problem is handled by the mathematics of that structure. The required expectation values have an exact classical expression, and evaluating it costs time that grows only linearly with the number of qubits. That is what makes it possible to train circuits from sixteen to sixty-four qubits inside a backbone at the scale of a full language model. Linear growth is the key property here. If the cost of wider circuits climbed steeply, the widths that give the best results would be out of reach.

Across downstream benchmarks, increasing circuit width raises the average score from forty-seven point six five to fifty-four point three zero. At sixty-four qubits, HyperQ exceeds the frozen backbone by four point seven one points, and its low-rank-adapted counterpart by three point six seven points. HyperQ is fine-tuned on twenty thousand prompt-response pairs. The classical baselines use two hundred thousand. So the quantum-augmented model is ahead on score while having been fine-tuned on a tenth as many examples.

The authors present this as support for token-conditioned circuit emission as a tractable architectural approach to quantum-augmented language modelling. The tractability comes from the exact classical expression. And the result that carries the argument is the trend: wider circuits, higher scores, on a tenth of the fine-tuning data.