WebHarbor: Docking Real Websites for Evolving GUI Agent Environments
Zhaoyang Wang, Qianhui Wu, Boyu Gou, Demi Wang, Jiacheng Zhang, Peichun Hua, Qize Li, Shi Qiu and 49 others
Published November 2, 2026 · on the feed November 4, 2026 · ▲ 27 · arXiv 2611.03634
The abstract
GUI web agents require realistic and interactive environments that remain stable during both evaluation and training. Live website evaluation suffers from changing content, access restrictions, and limited coverage of authenticated workflows, while existing synthetic environments for training often lack functional depth and visual fidelity. We propose WebHarbor, a suite of 153 executable and dockerized website environments that faithfully reconstruct the visual interfaces, content, and functional workflows of real-world web services under deterministic control. We build WebHarbor through task-driven human-agent collaboration, where environments evolve as new tasks expand their functional scope, coding agents implement the required changes, and human contributors audit visual fidelity, functional correctness, and task validity. The resulting suite comprises 3,518 tasks spanning multi-step and post-authentication workflows, where existing works often fail to cover. Beyond reliable evaluation, WebHarbor provides high-fidelity synthetic web environments for cost-efficient reinforcement learning (RL). We demonstrate that a Qwen 4B-level GUI agent trained with RL in WebHarbor generalizes to unseen tasks and performs competitively with leading open-source GUI web agents across four benchmarks. The code and data are publicly available at https://github.com/aiming-lab/WebHarbor.
The Brief
2:00A short podcast summary of the paper. Written and voiced by AI.
Transcript
WebHarbor. Docking Real Websites for Evolving GUI Agent Environments. By Zhaoyang Wang and fifty-six others. Published on the second of November.
Web agents that operate graphical interfaces need somewhere realistic to practise and be tested, and that place has to hold still. Live websites do not. Their content changes, access is restricted, and workflows behind a login are rarely covered. Synthetic environments built for training, on the other hand, often lack functional depth and visual fidelity. WebHarbor is offered as the middle ground: a suite of one hundred and fifty-three executable, dockerized website environments that faithfully reconstruct real web services, under deterministic control.
Each environment reproduces three things from its real-world counterpart: the visual interface, the content, and the functional workflows. On top of the environments sits a set of three thousand five hundred and eighteen tasks. They include multi-step workflows and workflows that happen after authentication, which the authors note existing work often fails to cover.
The suite was built through task-driven collaboration between humans and agents. New tasks expand what an environment has to support. Coding agents then implement the required changes, and human contributors audit the result for visual fidelity, functional correctness, and task validity. That is the sense in which the environments evolve: they grow as the tasks grow.
The obvious users are people evaluating GUI web agents, who get stable and repeatable sites in place of moving targets. The second audience is people training such agents. WebHarbor serves as a high-fidelity synthetic setting for cost-efficient reinforcement learning, and the authors demonstrate this directly. A Qwen 4B-level GUI agent trained with RL inside WebHarbor generalizes to unseen tasks, and performs competitively with leading open-source GUI web agents across four benchmarks.
Realism and stability usually pull in opposite directions for web agents. A real site is faithful but unpredictable, and a synthetic one is predictable but thin. WebHarbor tries to have both by reconstructing real services inside containers and letting human review keep them honest.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free