ControlScope: Workflow Revision and...

ControlScope. Workflow Revision and Reliability in LLM Agents. By Jingjie Ning and three others, from Carnegie Mellon University and Tsinghua University. Published on the twenty-eighth of September. No code repository is linked to it.

When a language model agent reviews a workflow that is already running, how much of it should it be allowed to revise? It could carry on with the code it generated, adjust only the data arguments of the next tool call, or discard the unfinished workflow and write a replacement. A better final score on its own says little about which repair an agent actually chose, or what happened once it chose.

ControlScope compares those three options from the same public execution state. The abstract labels them KEEP, ARG and FULL. The permissions are nested, each one containing the last, and that nesting is what lets the study separate the repairs an agent has available from the actions it actually selects. Reviews can happen once or repeatedly, and outcomes are followed forward through execution.

The study spans filesystem tasks, ALFWorld and AppWorld. The filesystem suite has twenty tasks, each with two source programs. The AppWorld panel comes from the official test split of V1: five hundred and eighty-five task instances drawn from one hundred and ninety-five scenario templates. Reviews come from two kinds of model draw, a reasoning reviewer and a fast one.

Filesystem results depend on the reviewer. Over three reasoning draws, FULL completes fifteen or sixteen tasks, against thirteen for KEEP. Over four fast draws, FULL completes between ten and thirteen, against thirteen. A student-record confirmation run fresh reproduces a repair that reads files in batches.

ALFWorld barely separates the three. On eighty-seven tasks across fifty-two scenes, fast panels score eighty-five, eighty-six and eighty-seven for KEEP, ARG and FULL. A reasoning reviewer on that cohort gives the same three numbers, at substantial review cost.

A second ALFWorld panel of one hundred and thirty-four tasks across four scenes goes the other way: one hundred and thirty-four, one hundred and thirty-four, and one hundred and twenty-seven. AppWorld shows only small net differences.

Replays explain some of this. Frozen replays of two failed file-organization runs expose viable replacements the agent had written, then interrupted by later revision. An offline comparison at the midpoint of source trajectories shows later reviews completing a repair that was insufficient. A five-call protection saves nineteen point four percent of logged model output and loses one success across twenty fresh source runs. And an argument-only shortcut shows that the broader sampled policy can overlook a cheaper edit that succeeds, one available in both operation sets.

Final success rates hide which door the agent walked through. ControlScope's contribution is to tie repair access to the choices an agent actually makes and to what execution does next. Seen that way, a wider licence to revise sometimes helps, sometimes ties, and sometimes costs a task that a narrower one would have kept.