MMedAgent-Plus: Scaling Multimodal Medical Agents across Modalities, Tasks and Workflows

MMedAgent-Plus. Scaling Multimodal Medical Agents across Modalities, Tasks and Workflows. By Ruoxin Wang and fifteen others, from Duke University, Stanford University, and eight other institutions. Published on the nineteenth of October. Its code repository is public and real: twelve source files.

Medical workflows often call for several analyses that depend on one another. Existing AI models, whether specialists built for one job or generalists meant for many, are not designed to coordinate a chain of analyses like that. Each does its own part, and nothing ties the parts together.

MMedAgent-Plus is a multimodal medical agent that uses tools. A vision-language planner sits at the centre, and around it are eighteen leading specialist models spanning twelve tasks and eight modalities. The planner does the coordinating. The specialists do the analysis.

Everything goes through a unified tool interface. Through it, the planner selects which specialists to call, executes them in sequence, and synthesizes a response grounded in what they returned. The planner is instruction-tuned on about two hundred and nine thousand samples drawn from thirty-two public medical benchmarks. Those samples cover single and multiple tool interactions, over single and multiple rounds.

Evaluation is split three ways on held-out samples: tool orchestration, response synthesis, and end-to-end execution. For orchestration, tool-call accuracy is ninety-one percent with a single tool over multiple rounds, ninety-eight point one percent with multiple tools over multiple rounds, and eighty point seven percent with multiple tools in a single round.

For synthesis, given correct tool outputs, scores run from fourteen point four to fourteen point eight out of fifteen for faithfulness, completeness and conciseness, across six hundred samples. End to end, with no tool outputs supplied, the agent invokes the required specialist in ninety point six percent of three hundred and fifty cases spanning seven tasks. Final-answer accuracy reaches zero point nine eight. That matches the dedicated leading specialist, against zero point two eight for the strongest baseline.

The design keeps expertise where it already lives, in the specialist models, and asks the planner only to route and to summarise. The split evaluation mirrors that division, checking the routing and the summarising separately before checking them together.