CANOPY: Adaptive-Granularity Evidence Compression for Multimodal RAG

CANOPY. Adaptive-Granularity Evidence Compression for Multimodal RAG. By Hyojeong Yun and two others, from POSTECH. Published on the first of October. No code repository is linked to it.

Multimodal retrieval-augmented generation pulls in text, tables, images and videos. Choosing how coarse a retrieved unit should be does not settle a second question: how much of each item to keep. Coarse units drag in irrelevant content. Uniformly fine selection can strip away the context needed to interpret the evidence. Existing compressors handle this trade-off with mechanisms built for one modality at a time, which leaves no shared procedure for deciding, region by region, how much to retain across items of different kinds.

CANOPY, short for Canonical Projection over Hierarchy, compresses evidence after retrieval at an adaptive granularity. Its central move is to represent every retrieved item as a hierarchy. Once items share that form, a single procedure can decide how much of each region to keep, whether the region is a passage of text, part of a table, or part of an image or a video. Granularity stops being one setting chosen in advance and becomes a decision made region by region.

A node encoder, fine-tuned on gold evidence, scores regions of the hierarchy against the query. Parent-relative refinement then compares those scores, so the framework can select several regions at different granularities. None of this node-level pruning requires calls to a language model.

Compression has a limit, though: it cannot recover evidence that was never retrieved. So a critic judges the accumulated evidence, and when it finds that evidence insufficient, it requests targeted follow-up retrieval. Newly retrieved items are compressed before they are added, so the loop can fetch more without letting the context grow unchecked.

Evaluation covers five question-answering benchmarks over a heterogeneous corpus of thirty-three million items. Across them, CANOPY reaches higher average answer accuracy than the retrieval baselines evaluated. Ablations indicate that the additional retrieval is what drives the main accuracy gains on multi-hop questions.

Compression earns its keep on cost. In the unrouted setting with Qwen3-VL-8B-Instruct, it reduces the evidence tokens fed to the reader by between fourteen point two and twenty-seven point seven percent, compared with the same iterative pipeline without compression, at comparable answer accuracy.

Two findings sit side by side here, and they are distinct. The compression is what saves tokens. The follow-up retrieval is what, by the ablations, earns most of the accuracy on multi-hop questions. Keeping less and fetching more are complementary halves of one framework.