NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale

NeMo-DCR. Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale. By Songlin Jiang and seven others, from Aalto University and NVIDIA. Published on the sixth of October. Its code repository is worked out: about one thousand five hundred source files, with a readme and a licence.

Agentic reinforcement learning separates training from rollout. That means every policy update has to travel to the rollout clusters before the next batch can begin, a weight synchronization step the authors call refit. At the trillion-parameter scale this is slow. Transferring a full checkpoint between two AWS regions takes eighty-seven point five minutes.

There is an obvious opening. Measurements of BF16 training show that only about one percent of weights change their stored values on any given step. Recent systems already exploit that sparsity, but each falls short somewhere: on placement, on exactness, or on efficiency. They reimplement placement rules, assemble full tensors, rebuild values arithmetically, or rely on a cross-cluster collective. And none of them fully recovers when a failure strikes in the middle of a refit.

NeMo-DCR, short for Delta-Compressed Refit, sends only the changes, and yet it is bit-exact. Receivers end up holding the same parameter and buffer bits they would have obtained from a dense refit. The design answers each of the three shortfalls in turn.

Placement first. Fixed affine mappings project changes from the training shards into the checkpoint's canonical coordinates. Residual conversion covers whatever changes those mappings do not, and the serving runtime's own native loader then places every change into receiver storage, so placement rules are not rewritten by hand.

Exactness next. Affine changes, where the projection and the loader preserve stored bits, travel as compressible XOR masks. All other changes travel as overwrites. Receivers apply both kinds in place. If a write is interrupted, a retry simply overwrites the partial result, and a joint commit binds the policy to the baseline that the next delta will be computed against. That is how a mid-refit failure is recovered. Efficiency last. Payloads stream during delta construction, through object storage or a relay tree, with no cross-cluster collective involved.

The results are given at change rates higher than the one percent typically measured: three percent and five percent. Even there, NeMo-DCR refits models from thirty billion parameters up to the trillion scale twelve to forty times faster than a transport-only reference that ships the full checkpoint. The headline case is a trillion-parameter refit over a relay tree at a three percent change rate. It takes one hundred and fifty seconds, instead of the eighty-seven point five minutes a full transfer needs.

Two properties usually pull against each other here: sending less, and guaranteeing the receiver ends up with exactly the right bits. NeMo-DCR keeps both, and adds recovery from partial failure on top. The authors' conclusion is that this makes refit practical for cross-cluster agentic reinforcement learning at the trillion-parameter scale, turning a step measured in an hour and a half into one measured in a couple of minutes.