LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

LongZero. Enhancing Long-Context Reasoning in LLMs from Zero Data. By Zhennan Wan and one other, from Tsinghua University. Published on the twenty-first of October.

Reinforcement learning is an effective way to strengthen how large language models reason over long contexts. The catch is the training data it needs. That data has to be long, diverse, challenging, and verifiable, all at once. Annotating it by hand is expensive. Data that occurs naturally is too scarce to sustain reinforcement learning. And existing synthesis methods lean on external corpora or on stronger teacher models, which become harder to find as the models being trained get stronger themselves. The supply problem grows with success.

A different line of work has already shown a way around this for short tasks. In reinforcement learning from zero data, a model learns from tasks it proposes to itself, and that approach has performed excellently. Its limitation is that existing methods target short-context tasks. LongZero carries the idea into long contexts: a method that lets a language model improve its own long-context reasoning without any external data at all. No outside documents, no stronger teacher.

The key move is a reversal of the usual order. Normally, questions and answers are written over documents that already exist. LongZero works the other way round. First, the model synthesizes a question and its answer, together with the supporting evidence, in a compact structured form. Only after that, as a separate step, is the evidence expanded into a long context spread across multiple documents. The long context is built around a task whose answer is already pinned down.

Two kinds of self-written task feed the training. One is multi-hop reasoning, where the answer depends on chaining pieces of evidence. The other is multi-round retrieval. The model writes both kinds itself, and it is then trained on them with GRPO.

The evaluation covers two language models across five long-context reasoning benchmarks. The comparison of interest is against reinforcement learning on data synthesized from external corpora, the very dependence LongZero is meant to remove. According to the experimental results, LongZero generalizes to real tasks rather than only to the ones it wrote for itself, and it outperforms both baselines. The authors read this as a viable and promising path to improving long-context reasoning.

Picture a student who learns to read long reports by first deciding what the answer should be and which facts support it, then burying those facts inside a thick stack of pages and practising on the result. That is the shape of the proposal. The question, the answer and its evidence exist before the long text does, and nothing in the loop comes from outside the model, so the data problem that grows with a model's strength is sidestepped rather than solved with more data.

LongZero. Enhancing Long-Context Reasoning in LLMs from Zero Data. By Zhennan Wan and one other, from Tsinghua University. Published on the twenty-first of October. This is the long version: the whole paper rather than the abstract.

Reinforcement learning has become a reliable way to make a language model reason better over long inputs, but only if someone supplies the right data. The paper lists four demands: the inputs must be long, answering must mean finding and combining evidence, the questions must be hard enough to produce a learning signal, and the answers must be checkable by a program. Human annotation of such data is expensive, and naturally occurring long-context questions are far scarcer than reinforcement learning consumes. Existing pipelines borrow from real documents, padded question datasets or a stronger teacher, and real-world text adds a confound: a model may answer from memory instead of from the context, and public text may overlap with the benchmarks.

Methods where a model proposes its own tasks already work for short problems in mathematics, code and games. Long context is harder, because a task is now a long input plus a question about it, and the model has to produce both. The obvious recipe, write a long document and then ask a question about it, inherits the very weakness it is meant to fix. A model that cannot yet reason reliably over long text cannot reliably write or verify questions about the long text it has just produced. The key observation is that the order of generation can be reversed. If the model first writes the question, the evidence and the answer as a short structured specification, the answer is fixed before any long text exists. The only remaining risk lies in expanding that evidence into prose.

Reasoning data come from fictional worlds. After a program samples a domain and a style card fixing the era and naming style, the model writes a world specification in strict structured form: a handful of named entities, with at least two people sharing a surname and two organisations with confusably similar names; four question chains of three to five hops that partly share bridge entities; and two or three distractor facts per chain that involve the same bridge entity but differ in role, time or attribute. Questions refer to intermediate entities only by description, while the answer must be a concrete name or number that appears word for word in a fact.

Next the model plans which of eight documents states each fact, spreading a question's evidence across different documents and placing each distractor away from the fact it imitates. It writes each document separately in its genre, stating its assigned facts exactly once. A program shuffles the eight documents into one context and keeps a question only if its answer still appears verbatim. The paper calls this the faithfulness gate. Retrieval data borrow the format of multi-round co-reference resolution: in a long conversation the user keeps asking for things like a song about systems, then asks for the third such song, prefixed by a random ten-character string. Every response in the conversation is the model's own, so it must tell apart its own near-duplicates. Because a program chose which needle is the target, the answer is correct by construction. Finally the model answers each reasoning question in small waves of two samples. A question is kept the moment it has drawn both a right and a wrong answer, and discarded if its first four answers agree. Reasoning rewards are all or nothing, with a whole-word match that refuses answers listing many candidates. Retrieval rewards give partial credit by string similarity, but only if the answer begins with the prefix.

Two models were trained: Qwen3 four billion Thinking, and Qwen3 eight billion. Each wrote and filtered its own data, and was compared against two baselines that differ only in their reasoning questions: HotpotQA questions padded with distractors through the LoongRL pipeline, and multi-hop questions written and verified by a much larger teacher over real web documents. Same configuration, same steps. On the smaller model the five-benchmark average rose from fifty-four point six eight to fifty-seven point four two, against fifty-six point five seven for the HotpotQA baseline and fifty-five point nine nine for the teacher. The comparison that tests the thesis is that last one. A stronger generator does not guarantee better training data, because questions written by another model need not match the policy's ability. The ablation adds a second finding: reasoning data alone and retrieval data alone each lost ground on some benchmarks, and the two-to-one mixture was the only variant above the base model on all five.

There is no limitations section. Its future work includes finding better ways to ensure the data are correct, and the appendix shows why. One case study shows a distractor that gives the same founding role to a different person with the same surname, so the documents support two answers. Because the model answered once with each name, the question passed the filter and entered training, where the reward penalises an answer the context supports. On the eight billion model the gain is about one point, the model fell below its base on LongBench version two, and all methods score near zero on one benchmark: a hundred and twenty of LongZero's two hundred responses there contain no answer, because inputs that long leave little room to finish thinking. Results are the mean of two runs, with no error bars, and several margins are under a point.

The idea to carry away is generation order. A model too weak to check a question about a long text it wrote can still check a short specification, so write the answer first and the haystack second. Fictional worlds remove memory as a shortcut, and self-filtering keeps the difficulty matched to the learner. The evidence that this beats outside data is real on the smaller model and thin on the larger one.