TRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning

TRIAGE. Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning. By Zhen Li and nine others, from InfiX AI and NVIDIA. Published on the fifth of October. Its code repository is public and real: about thirty source files.

Running reinforcement learning for large language models in low precision can speed it up substantially. The catch is that the learner and the sampler do not execute identically, and the discrepancy between them can destabilize policy optimization. The usual way to think about that mismatch is by its magnitude. The authors argue magnitude alone misses something: how the mismatch interacts with the direction of the policy gradient. Some update contributions are locally amplifying, others contracting, and size by itself cannot tell them apart.

Looking at native NVFP4 runs through that lens, they observe an early imbalance between the two amplifying regions, favouring updates with negative advantage and a negative gap. The tail tokens of those updates concentrate in a small fraction of response segments, before the mismatch spreads globally.

That observation suggests where to intervene, and when. TRIAGE is a direction-aware stabilization method. It diagnoses at the level of segments, so it can act on the small part of a response where trouble begins rather than on everything.

With that diagnosis, TRIAGE selectively rebalances policy-gradient updates, and it applies a bounded repair to whatever severe mismatch remains. The changes live in the optimization objective. Execution is left native: weights and activations stay in four-bit NVFP4 for the forward pass, on both the sampler and the learner.

Experiments use Qwen3-4B and Qwen3-30B-A3B. Optimization stays stable throughout the evaluated training horizon, and performance reaches full precision levels across five mathematical reasoning benchmarks. Meanwhile, native NVFP4 with TRIAGE delivers up to two point three times the rollout throughput of BF16.

The result reframes a precision problem as a direction problem. Instability is not handled by giving up the low-precision execution, but by finding which updates the mismatch is amplifying, catching them early in the few segments where they gather, and correcting the objective there.