When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

When Quantization Breaks Memory. Recurrent-State Write-Back in Low-Precision Temporal Inference. By Ismail Erbas and two others, from Rensselaer Polytechnic Institute. Published on the third of September. No code repository is linked to it.

Quantization is used everywhere to cut the computation and memory that inference demands. In a recurrent network it does something extra that is easy to miss. The quantized state is stored and then returned to the network at the next time step, so the rule used to store that state can alter every computation that follows. The paper names that rule recurrent-state write-back, and its claim is that the rule is a key determinant of how a low-precision recurrent network behaves, rather than an implementation detail beneath attention. The state-storage interface, on this account, is a central design consideration for quantized recurrent inference.

The effect is isolated in a compact GRU encoder-decoder built for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging, whose central task is estimating two lifetime parameters, one short-lived and one long-lived, from high-noise time-resolved fluorescence signals. Holding the trained model fixed and replacing continuous state propagation with deterministic four-bit state storage increases the estimation errors for those two parameters by approximately seventy times and three hundred times. The mechanism is stated precisely: failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network goes on proposing change. Error feedback, residual memory and direction memory each carry information from those suppressed updates across time, and recover accuracy without retraining.

Two further results mark out where the claim holds. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, which runs against the intuition that more bits are always safer, while matched training shows that compatibility with the state interface is something a model can learn rather than something it has by default. To test whether any of this is peculiar to the GRU, the post-training intervention is repeated in an independently trained LSTM. Coarse write-back reproduces the failure there, error feedback restores accuracy there, and state-specific interventions reveal greater sensitivity of the cell state than of the hidden state. The behaviour survives a change of architecture, which is what a claim at this altitude needed.

What makes this the kind of finding that costs people weeks is that all of it happens after training, to a model that was working. No weights changed. The quantization was applied the way quantization is normally applied, and a three-hundred-fold error increase followed from a decision about how to write a number back into memory. That is where the surprise sits: in the storage rule rather than in the bit width everyone was already watching. The recovery methods work without retraining as well, which is the practical half of the result.

When Quantization Breaks Memory. Recurrent-State Write-Back in Low-Precision Temporal Inference. By Ismail Erbas and two others, from Rensselaer Polytechnic Institute. Published on the third of September. This is the long version: the whole paper rather than the abstract.

Quantization is normally evaluated by asking how much numerical precision can be removed before a model loses accuracy. The paper's claim is that for recurrent networks this framing is incomplete, and the reason is structural. In a recurrent network the internal state is stored after each step and returned to the network to generate the next one, so the stored state is both a memory of what came before and an input to what comes next. Change how it is represented and you change the sequence of states the network visits. Storage stops being a passive encoding of a trajectory the network would have produced anyway, and starts helping to determine the trajectory.

The usual answer is quantization-aware training: fix the numerical representation before learning, and the network forms its dynamics around it. Two observations show that remedy is incomplete. Deployment can reverse the order, mapping a trained network to a coarser representation to satisfy hardware constraints without retraining, so every state is then written through an interface the network never trained with. And even when the constraint is present throughout training, coarse state precision can leave an accuracy gap that further optimization does not close. This second case is documented in the very system studied here: an eight-bit operating point was adopted because four-bit inference fell short of the required accuracy even with the constraint present during training, and nobody had identified the mechanism.

The paper names that mechanism. Recurrent-state write-back is the mapping from the state computed at one step to the value stored and presented at the next. Deterministic low-precision write-back rounds the computed state to the nearest representable level, so any proposed update that stays within the corresponding write boundary leaves the stored state unchanged and never enters the state the next step observes. That becomes consequential when the network repeatedly proposes small changes in the same direction. The computation keeps proposing movement while the stored trajectory stands still. The setting is a compact recurrent encoder-decoder estimating two fluorescence lifetimes from high-noise time-resolved signals, with a single layer, thirty-two units, and six thousand six hundred and twenty-seven trainable parameters.

The state-level statistics show what the failure looks like from inside. At that checkpoint, ninety-nine and a half percent of decoder updates fall inside the four-bit write boundary. Ninety-seven percent of decoder steps change no hidden unit at all, and on average a small fraction of one unit out of thirty-two changes level at each step. The network is computing new proposals continuously and storing almost none of them. There is a further detail that should worry anyone deploying such a model: the reconstructed sequence still looks fine, with a small pointwise error, while the lifetime estimate derived from its shape is severely wrong. A monitoring metric on the output sequence would not have caught this.

The scope is one task and two small architectures, which the authors address directly by repeating the whole post-training intervention in an independently trained network of the other common recurrent type. Coarse write-back reproduces the failure there, error feedback restores accuracy there, and targeting the two state variables separately shows the cell state to be substantially more sensitive than the hidden state. Beyond that the results are about compact recurrent models in a scientific imaging setting, not about large sequence models generally. And one finding cuts against tidy intuitions: precision sweeps show that increasing state precision can actually worsen a fixed recurrent solution, so more bits is not a reliable direction of travel.

What makes this the kind of result that costs people months is where it sits. All of it happens after training, to a model that worked, through a step nobody thinks of as part of the computation. The weights were untouched. The quantization was applied the way quantization is normally applied. And the useful output was destroyed by a decision about how to write a number back into memory, while the sequence the network produced still looked approximately right.