CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling

CLIFT. Conformal Self-Verification for Web Agent Training and Test-Time Scaling. By Yifan Zhang and five others, from Salesforce AI Research. Published on the fifth of October. No code repository is linked to it.

Open-source web agents can now carry out realistic browser tasks. Training them with reinforcement learning, though, still leans on weak supervision, and both of the usual signals fall short. Binary task success is too sparse to assign credit to individual steps. Judges built on frontier language models could fill that gap, but they are too expensive to call at every step, and nobody can assume they will be available at deployment. The agent needs a denser signal that does not depend on an outside judge forever.

CLIFT is a training and test-time scaling method built around one idea: conformal self-verification. The agent asks itself natural-language verification questions about its own rollouts. Not every question is trustworthy, so the method certifies which ones are, against a judge it can afford during training. The resulting bank of certified questions then serves twice: as a reward signal while training, and as structured evidence at test time.

During training, a Compositional Conformal Certifier keeps only those question signals whose URL-conditional evidence agrees with a training-time judge. It assigns signed trust weights through what the authors call polarity-aware lift. The resulting verifier score is blended into per-step rewards, and the blend is built so that it never subtracts from the judge baseline. So the judge still sets the floor, and the certified questions can only add a finer, per-step signal on top of it. A question earns a voice in the reward only if the evidence behind it agrees with the judge, and its weight can be positive or negative depending on which way it points.

At test time the certified bank is frozen and reused for Conformal Trajectory Selection, or CTS. The agent samples a greedy rollout and one or more diverse retries. The self-verifier summarises each URL trace. A conservative majority-vote rule then decides whether to swap away from the current incumbent rollout. No external judge is called at any point in that loop. The bank that was certified against a judge during training now stands in for one, and the conservative vote means a retry has to make a clear case before it replaces what the agent already had.

That single mechanism is tested in three settings. On WebArena Infinity, CLIFT reports state-of-the-art performance among open-source web agents. On VisualWebArena, a bank trained with the open model transfers to GPT five point five at test time, and reaches state-of-the-art performance under the canonical harness. On Online Mind2Web, no agent is trained on the benchmark at all. Translating the certified question bank is enough to improve a live-web agent in zero-shot evaluation.

The arc of the method runs from expensive to cheap. A costly judge is consulted during training, but only to decide which of the agent's own questions deserve trust. Once that is settled, the certified questions become a reusable training signal and, frozen, a test-time scaling signal that needs no judge at all. The authors position conformal self-verification as exactly that conversion, and the three settings, spanning an open agent, a frontier model and the live web, are the evidence they offer for its reach.