Adapting prior-data fitted networks for tabular anomaly detection

Adapting prior-data fitted networks for tabular anomaly detection. By Maximilian Bershtman and one other, from the Technion, Israel Institute of Technology. Published on the fifth of October. Its code repository is public and real: about thirty source files.

Deep features have transformed anomaly detection in images and video. On tabular data their impact has been less substantial, partly because strong deep representations for tables have been hard to come by. Prior-data fitted networks, or PFNs, have recently emerged as a promising source of them.

The question is how to adapt PFN representations for anomaly detection, and it is harder than it looks, for two reasons. First, no anomalies are available before deployment, so model parameters cannot be tuned with supervision. There is nothing labelled to learn the difference from. Second, the reference set that defines normal behaviour may itself contain the very anomalies it is supposed to reveal. The yardstick for normal can be contaminated by exactly what it is meant to catch.

The study works in two steps. First, take frozen TabPFN features and score each sample by its distance to its nearest neighbours in feature space. A sample that sits far from everything else is the one flagged. That alone already gives strong results. Second, use the reference set to fine-tune the model, so the resulting features separate normal samples from anomalies more cleanly than the frozen ones do.

Making the frozen route work involves two choices the authors pin down: which layers to use, and a feature-extraction procedure suited to the task. That fine-tuning-free approach is called ZEN. The fine-tuned method that builds on it is called FOCUS. Neither depends on labelled anomalies. The only data in play is the reference set, the same set the problem statement warned might be contaminated.

On the ADBench benchmark, ZEN reaches a higher mean AUROC than every baseline, and FOCUS improves on it further. The approach also generalizes across PFN models, so it is not tied to one particular network.

Two results stack. Frozen PFN features with a nearest-neighbour score are already enough to top the baselines on mean AUROC, and fine-tuning on the reference set adds more on top, all without anomalies available for supervision. The opening worry was that tables lacked strong deep representations. On this benchmark, PFN features answer it, and the answer holds across more than one PFN.