Harness Self-Evolution for Long-Horizon Terminal Agents

Harness Self-Evolution for Long-Horizon Terminal Agents. By Ruhan Wang and eight others, from Tencent HY LLM Frontier, Indiana University, and three other institutions. Published on the twentieth of October. No code repository is linked to it.

A long-horizon terminal agent can solve a task in one run and then fail that same task when the run is repeated. Stochastic errors spread through its actions and its context, and the result is a gap between what the agent is capable of and what it executes reliably. The capability is there. The dependability is not.

RiseHarness improves the harness instead of the model. Weights stay fixed, and the framework iterates on the scaffolding around them. Its attention goes to a particular kind of task, the ones the authors call flipping tasks: tasks with both successful and failed runs under one and the same configuration. Those are the informative cases, because the successful runs can serve as references against which the failures are analysed. When a task fails everywhere, or succeeds everywhere, there is nothing to compare. A task that does both under one configuration isolates what varied.

Evidence comes from two places at once. Agent trajectories supply what the model did. Internal harness traces from a runtime monitor supply what the harness did. Together they let the framework examine how the model's actions interact with command execution, with feedback, with context management, and with decisions about when a task is complete. A Harness Handbook links that evidence to harness behaviors and to source code, which is how recurring problems in mechanisms shared across tasks become visible.

Then RiseHarness looks outward. It searches other harness frameworks for implementations relevant to the problem it has found, and adapts them as design references for the current model and execution protocol. Candidate changes face behavioral checks and repeated task evaluation, and the versions with measured gains seed the next iteration. The loop therefore does two things at once: it pulls suitable mechanisms from different frameworks into a single harness, and it uses fresh execution evidence to steer the changes that follow.

The evaluation measures both task success and reliability across models. The authors ablate the two evidence sources, harness execution traces and the external research step, and they test transfer to unseen benchmarks while holding the model fixed. Transfer is the test of whether the harness has absorbed something general or only fitted the tasks it was iterated on.

What is unusual here is the unit of adaptation. RiseHarness treats a specific model as the thing a harness is tuned for, and asks what changes to the shared interaction mechanisms will make capabilities the model already has hold up across repeated runs. The target is not a better agent but a less fragile one.