The Linear Representation Hypothesis Needs a Group Action
Louie Hong Yao, Yuhao Li, Shengchao Liu
Independent Researcher · The Chinese University of Hong Kong
Published September 22, 2026 · on the feed September 24, 2026 · ▲ 3 · arXiv 2609.27158
The abstract
To make claims about representations that generalize beyond a particular trained model, we need to specify when two representations should count as equivalent. The Linear Representation Hypothesis is often discussed without making this equivalence explicit. Different notions of equivalence preserve different structures, so metrics, probes, and interventions that appear to study the same representation may in fact correspond to different hypotheses. We therefore argue that the Linear Representation Hypothesis is not one hypothesis but a family of claims distinguished by representation equivalence. We formalize this idea using group actions, specifying the representation object, the procedure that produces it, and the property ultimately asserted, while accounting for equivalences imposed by the model architecture. This framework clarifies how assumptions can change across metrics, reading points, and analysis stages, and we use it to audit common representation quantities and recent interpretability analyses.
The Brief
1:54A short podcast summary of the paper. Written and voiced by AI.
Transcript
The Linear Representation Hypothesis Needs a Group Action. By Louie Hong Yao and two others, from Independent Researcher and the Chinese University of Hong Kong. Published on the twenty-second of September. No code repository is linked to it.
Any claim about a representation that is meant to hold beyond one particular trained model has to say when two representations count as equivalent. The Linear Representation Hypothesis is often discussed without making that equivalence explicit.
The authors argue that the Linear Representation Hypothesis is therefore not one hypothesis but a family of claims. What distinguishes the members of the family is the notion of representation equivalence each one uses.
The reasoning starts from the fact that different notions of equivalence preserve different structures. Metrics, probes and interventions that appear to study the same representation may, as a result, correspond to different hypotheses. Two studies could both report evidence for a linear representation of the same concept and still be testing claims that are not the same claim. To make this precise, the authors formalize it using group actions. The formalism specifies three things: the representation object, the procedure that produces it, and the property ultimately asserted. It also accounts for equivalences imposed by the model architecture itself.
The framework's job is to bring assumptions into view. It clarifies how assumptions can change across metrics, across reading points, and across stages of an analysis, which is exactly where two studies of the same concept can quietly diverge. The authors use it to audit common representation quantities and recent interpretability analyses.
Put in one line, the hypothesis needs a group action. Saying that something is represented linearly is incomplete until you also say which representations count as the same one, and making that explicit is what separates a loose family of claims into distinct ones. It is a call for precision rather than a rejection of the hypothesis.
Paper Quest · 7 chapters · about 32 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Two studies, one concept, the same word for it. Are they saying the same thing?
You read two interpretability studies that both claim a concept is stored as a direction inside a language model. One used a probe to read it; the other added a vector to steer the model.
The road ahead
- 1 · The problem ~7 min Two studies can both say a concept is a direction in a model, yet nothing tells you whether they make the same claim.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Calling a representation linear is not enough: a claim must say which transformations leave it the same, what object it is, and how it is produced.
- 4 · Choose your own research ~3 min
- 5 · How it works ~3 min
- 6 · Field test ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.