Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems

Agent Mosaic. Recognizing Confidentiality Leaks in Agentic LLM Systems. By Daniel Ruiz and two others, from the University of Southern California. Published on the twenty-first of October. No code repository is linked to it.

An LLM agent can read several documents it has access to, and then disclose a sensitive conclusion that none of them states alone but that emerges when they are put together. This is the mosaic effect, and for agents working over confidential data it is a critical risk. The claim is that this leak is not hypothetical: it is an observed failure mode, occurring at non-negligible rates in the settings tested.

To measure it, the authors built thirty synthetic examples in cybersecurity, medicine and the military. From three LLM agents in controlled simulation these produced ten thousand eight hundred final answers, and live agents in two open-source harnesses produced one thousand four hundred and forty more. The queries were benign, designed to be answerable from a single document. Even so, the best model tested, GPT-5.6-Sol, disclosed a forbidden composite in seven point zero eight percent of its simulated answers, and disclosure rates were worse in the live harnesses.

Safeguards help, but not enough. Adding visible classification headers and explicit rules about aggregation substantially lowers pooled disclosure. Yet across all query intents, unsafe disclosures persist in thirty-two point three seven percent of simulated answers despite those safeguards. Refusing more does not settle it either: a significant share of refusals still reveal the prohibited conclusion while declining.

The claim is bounded by its setup. The examples are synthetic, thirty of them, across three domains. The simulation covers three agents, and the live tests cover two harnesses. The authors describe mosaic leaks as a failure mode in the evaluated settings, and they stop there. For each headline rate the abstract also gives a ninety-five percent confidence interval.

The results argue for stronger controls on what agents may infer and disclose, in sensitive environments where users hold mixed clearance. The core of it is that each document can be harmless on its own while the conclusion drawn from several of them is not, so labelling the documents and asking the agent to behave reduces the leak without ending it.