Building Auditable AI Text Detectors with Sparse Autoencoders

Building Auditable AI Text Detectors with Sparse Autoencoders. By Liam Dugan and five others, from the University of Pennsylvania and Google DeepMind. Published on the twenty-first of October. Its code repository is worked out: about forty-five source files, with a readme and a licence.

Detectors of AI-generated text are increasingly deployed where their outputs directly affect individuals: peer review, education, admissions, journalism, and content moderation. The strongest of them are fine-tuned neural classifiers, and their decisions are opaque. Transparent alternatives exist, classifiers built on stylometric and linguistic features, but they trail the neural detectors by wide margins. Accuracy and explainability currently sit on opposite sides.

The proposal is a pipeline that converts a fine-tuned neural detector into a classifier that operates on concepts a human can read. The neural detector is kept as the source of those concepts. What changes is the layer the final decision is made on: instead of opaque internal signals, the deciding classifier sees only features that have been given human-readable labels and have passed a check on how faithfully those labels describe them.

Sparse autoencoders, or SAEs, are trained directly on the detector's residual stream. Each SAE feature is given a label by auto-interpretation. Only the features whose generated descriptions pass a global fidelity threshold are retained. A lightweight classifier is then trained on those interpretable features and nothing else.

According to the authors, the SAEs recover latent concepts that correspond to highly specific and predictive linguistic tells of AI-generated text. The classifiers built on these features are highly interpretable. They also perform more strongly on out-of-distribution texts than either the neural detectors or the stylometric alternatives.

This is aimed at the settings listed at the start, where someone affected by a detector's verdict may ask why. A classifier whose inputs are named concepts can be audited in those terms, and on out-of-distribution text the abstract reports it outperforming both kinds of alternative rather than trailing them.