WISE-ATTA: When to Ask for Labels in Budgeted Active Test-Time Adaptation
Muhammad Huzaifa, Lea Schönherr, Thorsten Eisenhofer
CISPA Helmholtz Center for Information Security
Published September 29, 2026 · on the feed September 30, 2026 · ▲ 6 · arXiv 2609.37687
The abstract
Active test-time adaptation (ATTA) improves robustness under distribution shift by updating a deployed model during inference while selectively querying supervision. However, most existing ATTA methods implicitly assume that supervision can be requested for every incoming test batch, which can incur substantial annotation cost over long test streams. In this work, we introduce budgeted ATTA in which labels are available for only a fraction of test batches. This formulation shifts the central challenge from deciding what to label within a batch to deciding when supervision should be applied over time. To address this challenge, we propose a budget-aware approach WISE-ATTA that allocates supervision over the test stream based on lightweight signals computed online, prioritizing periods where supervision is likely to be most useful. When a batch is selected for supervision, we further employ a drift-based sample selection criterion that targets samples exhibiting ongoing, unconverged adaptation dynamics, enabling effective updates from a single labeled example. We evaluate this approach on synthetic corruptions (ImageNet-C) and natural distribution shifts (ImageNet-R/K/A). Across settings, WISE-ATTA achieves competitive or improved performance compared to recent ATTA methods while requiring substantially fewer labels. Overall, we find that the timing of supervision is a key, yet underexplored, aspect of active test-time adaptation. Code: https://github.com/Muhammad-Huzaifaa/WISE-ATTA
The Brief
2:28A short podcast summary of the paper. Written and voiced by AI.
Transcript
WISE-ATTA. When to Ask for Labels in Budgeted Active Test-Time Adaptation. By Muhammad Huzaifa and two others, from the CISPA Helmholtz Center for Information Security. Published on the twenty-ninth of September. Its code repository is worked out: about forty source files, with a readme and a licence.
Active test-time adaptation, or ATTA, keeps a deployed model robust under distribution shift. It updates the model during inference, and it selectively queries supervision as it goes. Most existing ATTA methods, though, carry a quiet assumption: that supervision can be requested for every incoming test batch. Over a long stream of test data, that assumption adds up to substantial annotation cost. Somebody has to provide all those labels, batch after batch, for as long as the model is running.
The paper's reformulation is budgeted ATTA, in which labels are available for only a fraction of the test batches. That small change moves the central question. It is no longer only which samples to label inside a batch. It is when, over time, supervision should be applied at all. A label spent early on an easy stretch of the stream is a label unavailable later, when it might have mattered more.
WISE-ATTA is a budget-aware answer to that question, working at two levels. Across the stream, it allocates supervision based on lightweight signals computed online, prioritising periods where supervision is likely to be most useful. Then, inside a batch that has been selected for supervision, it applies a drift-based sample selection criterion. That criterion targets samples whose adaptation dynamics are still ongoing and have not converged. Aiming at those samples is what allows an effective update from a single labeled example, rather than from a whole labelled batch.
The evaluation covers two kinds of shift. Synthetic corruptions come from ImageNet-C. Natural distribution shifts come from the R, K and A variants of ImageNet. Across these settings, WISE-ATTA achieves competitive or improved performance compared with recent ATTA methods, while requiring substantially fewer labels. The comparison is not that it wins outright everywhere. It is that it holds its ground on a much smaller annotation budget.
The broader point the authors draw is about timing. In active test-time adaptation, when supervision arrives turns out to be a key aspect of the problem, and one that has been underexplored next to the familiar question of what to label. A fixed label budget, spread thoughtfully over the stream, can do the work that a label for every batch was assumed to require.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Labels cost money, and the stream never ends. Start there.
Your deployed image model meets data unlike its training set. It keeps adapting itself during inference and now and then asks a person for a label to steer it.
The road ahead
- 1 · The problem ~7 min A deployed model that adapts on the fly can ask people for labels, but you can only afford a few, so where they go matters.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Decide when to spend a label using cheap online signals, then label the one sample that is still drifting, so one label is enough.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Score each batch with cheap signals, pick the promising batches, find the still-drifting sample, and update from its one label.
- 6 · The experiment ~6 min You’ll guess: Using substantially fewer labels than recent active test-time adaptation methods, how did WISE-ATTA's performance compare with theirs across settings?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.