Retrieval Augmented Generation Compositionality

Retrieval Augmented Generation Compositionality. By Ido Levy and two others, from IBM and the Weizmann Institute of Science. Published on the twenty-first of October. Its code repository is worked out: about eight hundred and fifty source files, with a readme and a licence.

Retrieval Augmented Generation, or RAG, grounds large language models in external knowledge. A modern pipeline stacks many techniques: embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification. Each of these is typically evaluated on its own. The claim here is that gains measured in isolation are not reliable predictors of how a full pipeline performs, and so new RAG techniques should be tested with the whole surrounding stack in place.

Three open questions are turned into three hypotheses, declared before the experiments. Do isolated gains survive composition? Do retrieval quality and answer faithfulness favour the same design choices? And do conclusions drawn on small corpora hold at large scale? To test them, the authors introduce LASAGNA, an open-source framework for controlled RAG evaluation. They run four hundred and fifty configurations across eight benchmarks, on corpora ranging from five thousand documents to two hundred thousand documents.

Each hypothesis produces a finding. First, individually strong techniques from different phases of the pipeline do not necessarily compose. Three retrieval techniques, judged one by one, predict a combined improvement of forty-seven percent. Used together, they deliver seventeen percent. Second, component gains stack differently for retrieval and for faithfulness, and optimising one can hurt the other. Third, the best composition shifts as the corpus grows. The combination that wins on a small corpus may not stay best at large scale.

Each finding undercuts a different shortcut. The first says a technique's isolated gain cannot simply be added to another's. The second says a pipeline tuned for retrieval quality is not automatically tuned for faithful answers. The third says a winner picked on a small corpus may not be the winner on a large one. Taken together, the authors conclude that measuring components one at a time is not a reliable guide to realistic performance.

The evidence covers the configurations, benchmarks, and corpus sizes the study tested, up to two hundred thousand documents. The findings are framed cautiously: techniques do not necessarily compose, optimising one target can hurt the other, and a small-corpus winner may not remain best. They establish that isolated gains can mislead, not that every combination fails, and the headline gap between predicted and delivered improvement is reported for one specific trio of retrieval techniques.

The paper ends with a request as much as a result. The authors call on researchers to evaluate new RAG techniques inside the full stack, and offer LASAGNA to make that easier. The arithmetic of the first finding is the clearest reason why: forty-seven percent expected from adding up the parts, seventeen percent from the parts working together.