Selecting The Most Informative Tokens in Natural Language Autoencoders
Federico Torrielli, Gianluca Barmina, Andrea Blasi Núñez, Amon Rapp, Luigi Di Caro, Peter Schneider-Kamp, Lukas Galke Poech
University of Turin · University of Southern Denmark
Published September 29, 2026 · on the feed September 30, 2026 · ▲ 16 · arXiv 2609.37040
The abstract
Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly. Which positions should an auditor inspect to understand a potential threat? We study this question across 4.7 million explanations on prompt injection and concealment. We compare signals from model computation with a ranker trained only on chat structure. Chat structure usually selects more relevant explanations than the computational signals, without requiring a model forward pass for position selection. On three of four datasets, explaining just 5% of positions retains nearly all of the success rate from explaining every position, where success means obtaining an explanation about the threat. The benefit varies with the audit task. We also show that pretrained verbalizers recover words that models have learned to conceal through fine-tuning, without additional verbalizer training. These results identify where auditors can concentrate explanation generation and show that useful explanations can extend beyond the model a verbalizer was trained to describe.
The Brief
1:54A short podcast summary of the paper. Written and voiced by AI.
Transcript
Selecting The Most Informative Tokens in Natural Language Autoencoders. By Federico Torrielli and six others, from the University of Turin and the University of Southern Denmark. Published on the twenty-ninth of September. Its code repository is worked out: about fifty-five source files, with a readme.
Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly, and that raises a practical question for an auditor. Which positions should be inspected to understand a potential threat? The answer here is that the structure of the chat itself is usually the better guide. A ranker trained only on chat structure tends to select more relevant explanations than signals taken from the model's own computation, and it does so without needing a forward pass through the model to choose positions.
The study covers four point seven million explanations, on prompt injection and on concealment. Success is defined as obtaining an explanation about the threat. On three of four datasets, explaining just five percent of positions retains nearly all the success rate of explaining every position. A second result concerns concealment directly. Pretrained verbalizers recover words that models have learned to conceal through fine-tuning, and they do it without any additional verbalizer training.
The claims are tied to two kinds of audit, prompt injection and concealment. The five percent result holds on three datasets of four, not on all of them, and the authors note that the benefit varies with the audit task. Chat structure wins usually, not always, against the computational signals. So the finding is a strong default for where to look, not a guarantee that the cheapest selection is always enough.
Two practical lessons follow for anyone auditing with these tools. Explanation generation can be concentrated on a small, cheaply chosen set of positions. And a verbalizer's usefulness can extend beyond the model it was trained to describe, reaching words a fine-tuned model was taught to hide.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every explanation costs real compute. Here the wall is a transcript with thousands of positions and a budget for a handful.
You audit a language model for threats like prompt injection. A tool can turn the model's internal activation at any token into a readable paragraph, but each paragraph costs about 130 generated tokens.
The road ahead
- 1 · The problem ~7 min A tool can explain any token of a transcript in plain words, but each explanation is costly, so an auditor cannot explain them all.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Rank positions by where they sit in the chat layout, before explaining anything; this needs no model run and spends a small budget well.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Rank every position by chat structure, explain only the top-ranked ones, then have a judge say whether each explanation is about the threat.
- 6 · The experiment ~6 min You’ll guess: On how many of four datasets did explaining just 5% of token positions keep nearly all the success of explaining every position?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.