SkinMind: Equipping Agentic RAG with Human-Inspired Graph Probing for Reliable Dermatology AI

SkinMind. Equipping Agentic RAG with Human-Inspired Graph Probing for Reliable Dermatology AI. By Jing Hao and thirteen others, from Ant Group, the University of Hong Kong, and four other institutions. Published on the sixth of November.

A dermatology system that can be relied on needs to reach verifiable evidence, and that evidence is spread across sources of very different kinds, some textual and some visual. Conventional retrieval-augmented generation, or RAG, usually performs a single retrieval from a flat corpus. That one-shot pattern limits it in three ways the authors name. It cannot adapt how fine or coarse its retrieval is. It cannot explore relations that take several hops to reach. And it struggles to bring multimodal clinical evidence together into one answer.

SkinMind is a multimodal agentic RAG framework built around what the authors call human-inspired graph probing. Instead of fetching once, retrieval becomes an iterative interaction between an orchestrator, which is the policy model, and a graph environment. The model seeks its evidence autonomously, step by step, rather than receiving a fixed bundle up front.

The orchestrator works through eight hierarchical graph-probing tools. With them it progressively identifies relevant entities, inspects information at the level of individual nodes, traverses relations and paths, and finally retrieves the source text and the visual evidence behind them. The authors describe this as progressive disclosure. Because the model decides what to open next, it chooses the retrieval modality, the direction of exploration, and the granularity of evidence for itself, building a subgraph of evidence specific to the query in front of it.

The graph it probes is SkinMind-KG, constructed for the purpose. Its sources are a corpus of dermatology guidelines and textbooks of sixty-two million tokens, along with four hundred and twenty-three thousand images. Construction is guided by a schema informed by dermatologists and by a conflict-resolution mechanism. The result holds over a million nodes and two million edges, uniting textual, relational and visual evidence in one place for retrieval.

Evaluation covers six dermatology VQA benchmarks under two policy-model settings. Compared with answering directly, SkinMind raises the average score by eight point seven points with GPT-5.4-mini, and by eight point eight nine points with Gemini-2.5-Flash. Across these settings it also consistently outperforms the competing RAG methods it was tested against.

Two contributions arrive as a pair. One is a way of retrieving: an agent that walks a graph with a graded set of tools and stops at the depth the question needs. The other is the graph that makes such walking possible, a dermatology knowledge structure where text, relations and images sit side by side. The authors offer them together as a foundation for multimodal reasoning in dermatology that stays grounded in evidence.