Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs
Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, Ran He
Nanjing University · University of Chinese Academy of Sciences
Published October 19, 2026 · on the feed October 26, 2026 · ▲ 40 · arXiv 2610.27103
The abstract
Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio-visual reasoning, yet cross-modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations. To address this issue, we propose Omni-Woodpecker (OWP), a training-free framework that shifts hallucination mitigation from output-level manipulation to conflict-conditioned representation rebalancing. Specifically, OWP first identifies cross-modal conflicts from complementary modality-specific views. It then employs orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. Finally, OWP adaptively rebalances fused representations by suppressing misleading priors and restoring ignored evidence before decoding. Extensive experiments demonstrate the effectiveness and strong cross-architecture generalization of OWP. Using Qwen2.5-Omni-7B as the backbone, OWP improves performance by 12.9 and 8.6 points on CMM and AVHBench, respectively, while consistently achieving gains of 0.67-2.25 points across four general OLLM reasoning benchmarks. Furthermore, OWP generalizes effectively to VideoLLaMA2-AV, improving performance by 4.6 and 3.9 points on the same benchmarks. Code is publicly available at https://github.com/ZiiJianLiu/Omni-Woodpecker
The Brief
2:11A short podcast summary of the paper. Written and voiced by AI.
Transcript
Omni-Woodpecker. Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs. By Zijian Liu and four others, from Nanjing University and the University of Chinese Academy of Sciences. Published on the nineteenth of October. Its code repository is worked out: about fifty-five source files, with a readme and a licence.
Omni LLMs reason jointly over audio and vision, and they have become capable at it. Cross-modal hallucination remains a major challenge for them. Most existing mitigation works at the output level, and the authors argue that this misses where the trouble sits. Inside the fused representations, two things are entangled: grounding evidence drawn from the inputs, and priors the model has learned. Adjusting outputs leaves that entanglement untouched.
Omni-Woodpecker, or OWP, moves the intervention inward. It is a training-free framework that replaces output-level manipulation with representation rebalancing, and the rebalancing is conditioned on conflict. Where the modalities disagree, the representation is adjusted so that evidence and prior carry appropriate weight before the model decodes anything.
That happens in three steps. First, OWP identifies cross-modal conflicts by looking at complementary views specific to each modality. Second, it uses orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. Third, it adaptively rebalances the fused representations, suppressing priors that mislead and restoring evidence that had been ignored. All of this is done before decoding, and none of it requires training.
With Qwen2.5-Omni-7B as the backbone, OWP improves performance by twelve point nine points on CMM and by eight point six points on AVHBench. Across four general OLLM reasoning benchmarks, it achieves consistent gains of between zero point six seven and two point two five points. The method also carries over to a different architecture: on VideoLLaMA2-AV it improves those same two benchmarks, CMM and AVHBench, by four point six and three point nine points.
Practitioners running an omni model they cannot or would rather not retrain are the obvious audience, since the method leaves training alone and works on hidden states at inference. The results cover two backbones, and the code is publicly available.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model can hear silence and still say the dog is barking. The question is why.
You ask a model that hears and sees a video: is the dog barking? The audio is silent, but the model says yes. Seeing a dog has overpowered what it heard.
The road ahead
- 1 · The problem ~6 min An audio-visual model can hear silence yet still say the dog is barking, because a learned expectation drowns out the evidence inside it.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Hallucination comes from evidence and priors tangled in fused states, so find the conflict, split evidence from prior, and rebalance before decoding.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Three stages on a frozen model: diagnose the conflict from views, decompose evidence and prior orthogonally, then rebalance the fused state before decoding.
- 6 · The experiment ~6 min You’ll guess: With Qwen2.5-Omni-7B as the backbone, how many points did OWP add on the CMM hallucination benchmark?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.