MMSearch-OPD: Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents

MMSearch-OPD. Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents. By Huanyao Zhang and eleven others, from Peking University, the Hong Kong University of Science and Technology, Guangzhou, and four other institutions. Published on the fifteenth of October. No code repository is linked to it.

Multimodal deep search is moving from short queries toward long-horizon investigations that keep acquiring evidence as they go. Training agents for that work runs into two limits. There are too few challenging training questions, and the feedback on the process is insufficient.

MMSearch-OPD takes on both, combining controllable data refinement with training that joins reinforcement learning and on-policy distillation. The title compresses the plan: easy made hard, and hard made learnable.

The Refine pipeline starts from existing simple multimodal questions and performs targeted expansion of the reasoning chains derived from them. Parameterized transformations control how deep the search must go and what evidence it needs across modalities, while the existing image and question pairs are reused.

Training then combines two signals. Answer rewards and lightweight behavior penalties guide reinforcement learning. Alongside that, the reward gap modulates token-level guidance from a teacher on the student's imperfect trajectories. A distillation coefficient decays exponentially, so the student leans on the teacher less and less as training proceeds.

Experiments show gains from data refinement, and further improvements from joint training after supervised fine-tuning. The authors take this as a sign of stronger long-horizon multimodal deep search capabilities.

Each half of the title maps onto one half of the method. Refinement makes easy questions harder and deeper without needing new images. Joint training with a fading teacher makes those harder questions something a student can actually learn from.