UnAct: Gradient-Free Unlearning via Targeted Activation Intervention

UnAct. Gradient-Free Unlearning via Targeted Activation Intervention. By Saeed Abdul Muizz and three others, from the National Institute of Technology Srinagar. Published on the third of October. Its code repository is worked out: about sixty-five source files, with a readme.

Machine unlearning tries to remove the influence of designated training data from a model that has already been trained, without paying to retrain it from scratch. Retraining is the reference point: a model that never saw the data at all. Everything else is an attempt to get close to that cheaply.

Retrain-free methods exist. Selective Synaptic Dampening, or SSD, and its label-free variant LFSSD avoid full retraining. But they still need backpropagation, and they still compute parameter importance over the entire dataset. That is a heavy requirement to carry into a deletion request.

The authors pose a narrower and more practical question. What happens when a deletion request arrives carrying only a few images of the class to be forgotten? Not the whole class, not the full dataset, just a handful of examples and an instruction to forget what they stand for.

UnAct is a class-unlearning method built for exactly that situation. It is gradient-free, and the only thing it needs is forward passes over the images to be forgotten. Three things are absent by design: no gradients, no labels, and no retained data from the rest of the training set. Everything the method knows about what to remove comes from watching how the network responds to the forget images.

The procedure is short. UnAct scores late-layer units by their responses to the forget images. It then attenuates the most responsive connections. And it repeats that cycle, for up to twenty rounds. Each round looks again at what the forget data still activates and turns it down, and every step uses forward passes alone.

The main tests use ResNet-18 trained on CIFAR-10, CIFAR-20, and CIFAR-100. When entire classes are forgotten, UnAct is competitive with SSD and LFSSD. The difference shows when forget data is scarce: unlike them, UnAct never collapses the network. Across all tested sizes, its retain accuracy stays within two point five points of retraining. SSD and LFSSD, at their full-class operating points, lose up to eighty-six points on some classes.

The sharpest case is five forget images on CIFAR-10. There, UnAct's distance to retraining is zero point two one points. For LFSSD it is sixty-seven. For SSD it is ninety. One might suspect SSD simply had the wrong threshold for so few images, but re-selecting its threshold at each size, with an oracle, does not close that gap.

A preliminary transfer to ViT-B/16 points the same way. UnAct's distance to retraining is eleven point five, against thirty-three point seven for SSD. And when SSD computes its importance at request time, an UnAct request runs nineteen times faster.

What sets UnAct apart is what it does without. The existing retrain-free methods were built around full-dataset importance and backpropagation, and they hold up when a whole class is available to forget. A request with a handful of images is where they break, sometimes badly enough to collapse the network. In that same regime, a loop of forward passes and dampening stays close to the model that retraining would have produced, and it gets there faster.