AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

AgentSafeEvo. Adversarial Co-Evolution for LLM Agent Safety. By Chenxu Wang and six others, from the Shanghai Artificial Intelligence Laboratory, Fudan University, and two other institutions. Published on the fourteenth of October. Its code repository is worked out: about one thousand source files, with a readme and a licence.

Large language model agents complete complex tasks through many turns of interaction with external tools. That openness is also an attack surface. Malicious content can be embedded in a user's query, in a tool's description, or in what the environment returns, and it can manipulate the agent's decisions into unsafe actions. Adversarial co-evolution has emerged as an effective way to strengthen LLM safety, but existing work mostly targets safety at the level of language, or indirect prompt injection in web tasks. Agents that use tools across many kinds of task are left less covered.

AgentSafeEvo is a co-evolutionary reinforcement learning framework for agent safety, built on a game between an attacker and a defender. The game spans diverse tasks and diverse attack types, so the pressure on the defender is broad rather than confined to one setting.

Both sides learn. The attacker learns to construct adversarial agent tasks that expose weaknesses in how the defender uses tools. The defender learns to complete tasks safely while under those attacks. On top of the game sits learnability-guided task weighting. It uses how much the defender's reward varies within a task to adjust how much that task contributes to the updates of both attacker and defender.

Experiments cover benchmarks for agent safety and benchmarks for general capability. Across them, the method improves adversarial robustness while maintaining the agent's general capabilities. The claim has two halves, and both matter: a safer agent that had become less useful would not be the result being reported.

The design rests on two learners pushing on each other. One side searches for the tasks that make the defender's tool use go wrong; the other learns to finish those same tasks safely. The weighting decides which tasks count most in that exchange, and the reported outcome is safety gained without capability lost.