miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning

miniTE. Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning. By Zhen Li and eight others, from InfiX AI and NVIDIA. Published on the seventh of October. It links to a code repository that is no longer there.

Native NVFP4 promises to speed up reinforcement learning for large language models. The difficulty is stability. In this kind of training the learner and the sampler often run in separate engines, and getting stable learning across the two is hard. The authors show that matching the model weights and the numerical formats between engines is not enough. Two engines holding the same weights in the same format can still evaluate the policy differently, because what matters is how the numbers are actually executed in each engine.

miniTE is a lightweight framework aimed at that execution gap. It brings together three pieces: execution diagnostics, cross-engine numerical alignment, and efficient native W4A4 GEMM computation, meaning four-bit weights and four-bit activations, for reinforcement learning on language models.

Two properties define how it sits in a training stack. It improves the consistency of numerical execution without modifying the reinforcement learning objective at all. And it reduces the implementation overhead that would otherwise stand between a team and the benefits of native low-bit computation.

The experiments are long-horizon reinforcement learning runs on Qwen3-30B-A3B. Training is stable, and benchmark performance lands close to BF16. In evaluation quality it outperforms naive Transformer Engine NVFP4.

Throughput is measured on a single NVIDIA B300 GPU node, for complete learner updates. The optimized learner reaches thirty-three thousand five hundred and forty tokens per second. That is one point three nine times the throughput of BF16, and one point four seven times that of the Transformer Engine NVFP4 baseline.

The lesson the authors draw is that low-bit reinforcement learning fails less on format than on execution: identical weights are not identical behaviour when two engines compute them differently. The framework, kernels, diagnostics and training configurations are released to make native low-bit reinforcement learning easier.