Opera: A Verbal Critic Framework for Long-horizon Coding Agents

Opera. A Verbal Critic Framework for Long-horizon Coding Agents. By Kai Mei and nine others, from Salesforce AI Research, Rutgers University, and one other institution. Published on the eighth of October. Its code repository is worked out: about one thousand five hundred source files, with a readme.

A coding agent working on a long task benefits from being corrected at the right moment. But feedback can fail. It can be ineffective, or even harmful, when it misjudges the work in progress or does not address the real problem. Existing critics concentrate on two jobs: evaluating trajectories and generating feedback. What they rarely do is follow up, watching what happens once the feedback has been delivered.

Opera is a verbal critic framework built around that follow-up. It treats every correction as a persistent note, and it keeps following that note until the problem it diagnosed has actually been resolved. A correction, in other words, is not finished when it is said. It is finished when it has worked.

The framework moves through four steps. It decides when to review, using triggers that are both periodic and driven by events. It diagnoses issues with typed operators. Before anything is delivered, it audits the feedback against evidence that is visible. And afterwards it tracks the agent's next actions, so it can tell mere compliance apart from genuine resolution. An agent that nods along and an agent that has fixed the problem are not treated as the same.

Used as a critic at test time, Opera raises the resolve rate of agents without a critic by up to twelve point four percentage points on Terminal-Bench 2.1, up to fifteen points on a SWE-Bench Pro subset, and up to eight point nine points on DeepSWE v1.1. Those figures hold across four policy models. On all three benchmarks it has the highest mean resolve rate among competitive critic baselines, and it also helps when a policy model critiques itself.

The second use is for training. Rollouts guided by Opera supply data that is approximately on-policy. Fine-tuning Qwen3.5-9B on them improves its resolve rate on held-out SWE-Bench Pro repositories by ten point two percentage points, with no critic present at inference. That matches fine-tuning on rollouts from a stronger model. It also keeps its performance when the harness changes from Openhands to Terminus-2, a switch that substantially degrades the model trained on the stronger model's rollouts.

The organising idea is accountability for advice. Opera does not stop at saying what is wrong. It checks its own feedback before giving it and then holds the note open until the agent's behaviour shows the problem is gone, and the trajectories that result turn out to be useful teaching material as well.