Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition

Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition. By Guray Ozgur and four others, from the Technical University of Darmstadt and the Fraunhofer Institute for Computer Graphics Research. Published on the second of November.

Vision Transformers perform strongly in face recognition, but they infer at a fixed depth, and that limits where they can be deployed when resources are tight. Stopping early would help, except that a standard ViT gives its intermediate layers no direct objective. Those layers are supervised only through backpropagation from the end, so their representations are never optimized to form usable identity embeddings. Exit halfway and what comes out was not trained to be a face embedding at all.

SD-ViT is a training framework that makes every block depth a usable exit, by making every depth produce discriminative embeddings. A deployment then picks a fixed exit depth to fit its budget. The ingredient is a stochastic auxiliary signal: at each training iteration, one intermediate block is selected at random, passed through the same embedding head as the final layer, and trained with a weighted margin-penalty softmax loss. The objective amounts to deep supervision, sampled stochastically, through a shared classifier.

Sharing is the point. Both loss terms run through the same head and the same classifier, so intermediate tokens are trained to map into the same embedding space as final-layer tokens. For face recognition this has a concrete payoff, because matching works by similarity. A probe image processed at an early exit can be compared directly against a gallery of embeddings computed at full depth. There is no need to recompute the gallery, and no adaptation layers to bridge the two depths. The gallery-consistent part of the title refers to exactly this: the reference set stays as it is, whatever depth the probe stops at.

On established benchmarks, SD-ViT stays within zero point four IJB-C points of the same backbone trained conventionally at full depth, so making every depth usable costs very little at the end of the network. Its early-exit probes recover more than ninety-six percent of their own full-depth performance at fifty percent of the computational cost, without any change to the reference gallery.

One more result bears on deployment. A single verification threshold, calibrated at full depth, remains usable from quarter depth onward, so the operating point does not have to be retuned for each exit.

Early exit for face recognition only works if an early embedding and a late one can be compared as equals, and that requirement shapes everything here. One shared head, one shared classifier, and a randomly chosen block trained through them at each step: the result is a family of exits that all speak the language of the existing gallery, with a threshold that carries across most of the depth.