Adapting Without Gradients: Affine Statistics Transport and What Its Certificate Can Tell You

Adapting Without Gradients. Affine Statistics Transport and What Its Certificate Can Tell You. By Salim Khazem and one other, from the Talan Research Center. Published on the thirty-first of August. No code repository is linked to it.

Test-time adaptation usually assumes a model's parameters can be updated while it is serving. That assumption fails in several real deployments: on accelerators built only for inference, for frozen or third-party models, and where memory is tight. Standard configurations based on BatchNorm can also go quiet on architectures that have no BatchNorm at all. The setting studied here is adaptation when the learned model must stay frozen: no weights change, at any point, for any batch. Whatever adaptation happens has to happen somewhere other than the parameters.

CASTER is a gradient-free answer. It stores the statistics of each source class inside a discriminative subspace. When a target batch arrives, it estimates a single affine transformation, shared across classes, from that batch's moments, and then analytically transports the source class distributions before classifying. In plainer terms, rather than bending the model toward the new data, it moves its picture of the old classes to where the new batch suggests they now sit. No backward pass, no optimizer state, and no stored bank of source features. All of it is arithmetic on statistics.

Transport, though, is not always safe, and the second half of the work is about knowing when. On ImageNet-C, where a batch holds only sixty-four samples for one thousand classes, transporting unconditionally costs twenty-one point two points of top-1 accuracy. So the authors introduce an empirical certificate, a ratio of residual to margin, that estimates transportability. Across three hundred and seven evaluation cells, every transport that lost more than ten points had a certificate value above three point nine. A high value, in other words, is a warning sign to take seriously. Benign and destructive regimes are still not perfectly separated, and the authors say so plainly rather than claiming a clean line.

Across four backbones and seven datasets, CASTER beats k-NN on identical frozen features in twenty-seven of twenty-eight backbone and dataset settings, while keeping a median of eighteen times less state.

Gating on the certificate turns the average effect of unconditional transport, a loss of three point three five points, into a gain of one point six nine points. Performance stays within zero point three points of the best threshold across a broad range of thresholds, so the choice of cutoff is not delicate.

One last test checks whether the certificate is general. Applied to Tent, it accepts only four point three percent of updates and preserves just zero point six percent of the gain Tent could have delivered. The certificate is specific to the mechanism it was built for.

Half of this work is about its own limits. Beyond the wins, twenty-seven settings out of twenty-eight with far less state to carry, the authors show where it breaks, build a signal to catch the breakage, measure how good that signal is, and then show that it does not transfer to a different adaptation method. CASTER is offered as a lightweight adaptation mechanism for deploying frozen models, and it arrives with an explicit account of where its safety signal is informative, and where the same signal stops telling you anything useful.