The LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems

The LLM-Attacker. A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems. By Georgios Politis and one other, from Setloop dot io. Published on the fifteenth of October. Its code repository is worked out: about two hundred and fifty source files, with a readme and a licence.

A split language model keeps the input text on a client, but sends intermediate representations to a server. Attacks exist that reconstruct the text from those messages. Evaluating how private such a system is turns out to be hard to compare from one study to the next, because attackers may see different messages, know different model weights, or be tested on different datasets with different recovery measures. Two privacy numbers can look comparable while answering different questions.

LLM-Attacker is a framework that records those choices, and lets researchers check an evaluation, compare it with others, and extend it. The scoring rule is strict about what counts as a pass: a defence cannot pass without a linked undefended reference to measure against. A defence is judged by how much it changes recovery compared with no defence at all, not by a number standing on its own. Reports also identify which attacks remain untested, so what an evaluation leaves out is stated alongside what it covers.

The records do practical work. Compatible attacks reuse the recorded messages, so different attacks can be run against exactly the same intercepted material. Their predictions are saved, so anyone can score them independently, without rerunning the attack. The framework reports implementation checks separately from how effective an attack was measured to be, so a correct implementation and a strong result are never confused.

In a controlled GPT-2 experiment, giving the attacker gradients returned during training raised token recovery from ninety-four point two percent to ninety-seven point three eight percent. Exact-document recovery rose from thirteen point seven one percent to thirty-seven point seven seven percent. The ninety-five percent interval for the token gain lay above a threshold of two percentage points that was fixed before testing.

That experiment is shared with a companion study. Here its role is not to report a new attack, but to test how the evaluation responds when attacker access changes. The question is whether the framework registers a change in what the attacker can see. Further studies measure how the choice of corpus and compression defences affect recovery, compare an adapted published attack with its reference implementation, and test the measurement code itself.

The paper closes on a standard rather than a score. Each privacy claim, in its view, should state the attacker, the dataset and the scoring rule, report its controls, and identify the attacks it did not test. The framework is the machinery for making that reporting routine.