Safety of Latent Communication in Multi-Agent Systems

Safety of Latent Communication in Multi-Agent Systems. By Muhammad Huzaifa and two others, from the CISPA Helmholtz Center for Information Security. Published on the thirtieth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

Agents in a multi-agent system can talk to each other in text, or they can exchange information directly in the space of their internal representations. That second route, latent communication, saves tokens, computation, and latency. It works through lightweight trainable links, which map the sender's representations into the receiver's input space. The finding concerns those links. Even training them on benign data can make the system comply with harmful requests more often than text-based communication would, while the safety-aligned agents at either end stay exactly as they were. Nothing about the agents changes. Only the channel between them does, and that alone is enough to shift how the system as a whole behaves when asked to do harm.

An attacker can make it worse. One way is to optimize the links on pairs of harmful queries and responses. Another is to poison training data that would otherwise be benign. The authors also build a reinforcement-learning attack that rewards harmful compliance alongside good performance on benign tasks, and it needs no harmful target responses at all. Across three communication topologies and four safety benchmarks, that attack pushes the mean harmful-compliance score from twenty-seven point nine, with benignly trained links, up to seventy-six point nine. Next to direct supervised optimization, it also keeps higher average accuracy on two benign utility benchmarks. The system stays useful while it becomes less safe.

The damage is not one-way. Turning the same reward machinery toward safer behaviour repairs compromised links, and substantially reduces harmful compliance across every attack evaluated. That repair, like the attacks, touches only the links, never the agents. The same lever that an attacker pulls can be pulled the other way by a defender, using rewards rather than retraining the models at either end. What the abstract establishes is the setting of these results: links trained between safety-aligned agents, three topologies, and four safety benchmarks.

Aligning each agent on its own, then, does not settle the question. A small trained component sitting between two safe models can undo their safety, or restore it, and in both directions the agents themselves are left untouched. Safety alignment, the authors conclude, has to treat the multi-agent system as a whole.