SkillDRE: Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback

SkillDRE. Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback. By Pengyu Zhu and four others, from the Beijing University of Posts and Telecommunications, North China Electric Power University, and one other institution. Published on the twenty-sixth of September. Its code repository is public and real: about twenty-five source files.

Agent skills bundle instructions, executable code and task-specific resources into reusable packages, and agents can improve them using feedback from execution. The same loop is open to attackers. A malicious skill can be evolved, too, to become more effective and harder to detect. Defences come in two stages, though, and they pull an attacker in different directions. A candidate skill can pass the scanner that inspects it before execution, and still fail to reach its goal once runtime defences are active. A revision that fixes the runtime failure can then trip new findings from the scanner. Optimizing against either stage alone leaves the other one standing.

SkillDRE is a fully automated red-team framework that evolves complete malicious skill packages against both stages at once, through a dual-stage feedback loop. It does not choose between satisfying the scanner and surviving the runtime defence. It alternates between them until one candidate does both. The point of building such a thing is to show what an adaptive attacker could achieve, so that the defences can be measured against it rather than against attacks that were never shaped to beat them.

It begins with a benign task and the skills associated with it. From these, SkillDRE constructs and validates, on its own, a task-conditioned malicious objective and a verifiable judge rule for deciding whether that objective was met. Both are then held fixed while the skill implementation evolves, and the evolution also has to preserve the skill's legitimate task capability. Two kinds of feedback drive it. Scanner-guided evolution works against the pre-execution stage. Runtime-guided refinement uses the outcomes observed when the skill runs under a runtime defence. Every runtime-guided revision goes back to the pre-execution stage to be rescanned and optimized further before it is executed again. That return trip closes the loop across the two stages.

The evaluation uses SkillsBench, across four victim models. SkillDRE reaches an average attack success rate of forty-five point two eight percent, which exceeds the strongest baseline by forty point three percent. Its final submitted skills draw no findings from SkillScan, and they largely preserve performance on the benign task. Put together, those three results describe a skill that works as an attack, passes inspection before it runs, and still does the job it claims to do, which is the combination the dual-stage loop was designed to reach.

Two conclusions come out of this, one for each side. For red teamers, feedback from both defence stages together is a useful learning signal for adaptive red teaming. For defenders, the warning is sharper: testing either stage in isolation can miss an attack that only exists because it was shaped against both.