What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document

What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document. By Georgios Politis and one other, from Setloop dot i o. Published on the second of October. No code repository is linked to it.

Split learning lets a client train a language model on a server without sending its text. The client runs the first layers itself and sends the server only their output, a vector of numbers for each token. During training, the server sends gradients back. The claim here comes in two parts. An observer at the split can rebuild most of the client's text from this traffic. And how much the gradients add to that depends heavily on how the leakage is counted.

The main experiment is on GPT-2, with an attacker who holds only the publicly released weights of the client's layers. From the activations alone, that attacker recovers ninety-four point two zero percent of tokens. Seeing the gradients as well raises this to ninety-seven point three eight percent. That is three point one seven percentage points more, with a ninety-five percent interval running from two point seven two to three point six four.

Counted by document, the difference is much larger. Without the gradients, the attacker rebuilds thirteen point seven one percent of thirty-two-token documents exactly. With them, the figure is thirty-seven point seven seven percent. The reason lies in the counting rule: a document only counts when every token in it is right, so a small gain per token turns into a large gain per document.

The counting choice changes how a defence looks, too. Secret mixup blends each outgoing vector with a decoy. It stops the attacker from rebuilding almost any document exactly, and yet the attacker still recovers eighty-three to ninety-one percent of tokens. Judged per document, the defence looks nearly complete. Judged per token, most of the text still gets through.

A second experiment, on GPT-2 and Qwen3-0.6B, has the server train only a run of consecutive layers. The layer at which that run starts changes both model quality and leakage, even when the run's length is fixed.

The numbers carry their conditions with them. The token and document figures come from GPT-2, against an attacker holding the public weights of the client's layers, on documents of thirty-two tokens. The defence result concerns one defence, secret mixup. The finding about where the server's layers start covers two models, GPT-2 and Qwen3-0.6B, with the length of the run held fixed.

Two recommendations follow from the authors' results. Report leakage both per token and per document, since each count can make a result look very different from the other. And treat what a split model sends as being as sensitive as the text itself.