OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents

OS-Helix. Unifying GUI Interaction and Code Execution in Native Computer-Using Agents. By Bowen Yang and ten others, from the University of Science and Technology of China, the Shanghai AI Laboratory, and six other institutions. Published on the fifteenth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Real desktop automation is neither purely visual nor purely programmatic. An agent has to understand the interfaces a user sees, and also manipulate files, scripts and system state efficiently. One might hope to get both by handing a GUI agent a code tool through its prompt. The authors' preliminary experiments say otherwise: naively adding a code tool made a GUI agent worse. Their conclusion is that routing between interface and code cannot come from composing tools at the prompt level. It has to be internalised through training, on high-quality hybrid data.

OS-Helix is an end-to-end trained multimodal agent for using a computer, and it places GUI actions and code execution in a single action space. Code is not a separate tool the agent calls out to. Clicking through an interface and running a script are both moves it chooses among.

Most of the work is in building the supervision. Application-artifact composition pairs desktop applications with authentic file artifacts, which instantiates executable Ubuntu states. A decoupled pipeline then generates instructions and evaluators separately, producing tasks grounded in the environment whose final states can be verified no matter which modality solved them. For hybrid data that property matters: a task finished through the interface and a task finished through a script are checked the same way.

Training begins with a cold start, SFT on filtered hybrid data, and continues with online RL in live desktop environments. For that stage the authors propose Step-balanced GSPO, which aggregates importance weight by action step rather than by raw assistant tokens. Without that change, long code actions would dominate optimisation over a mix of very different kinds of action.

The main benchmark is OSWorld-Verified. OS-Helix-27B and OS-Helix-35B-A3B improve over their matched Qwen3.5 hybrid baselines by thirty-five point zero percent and thirty-three point seven percent, in relative terms. Reinforcement learning then adds more: OS-Helix-27B-RL improves by a further forty-two point nine percent, reaching a success rate of sixty-six point four four percent.

Ablations are consistent with contributions from three sources: learned coordination between GUI and code, scalable verifiable supervision, and optimisation balanced by action step. They also demonstrate transfer across platforms, on WindowsAgentArena.

Taken together, the paper treats coordination between interface and code as a skill to be learned rather than a tool to be bolted on. It then builds what that learning needs: environments to act in, tasks that can be verified, and an optimiser that keeps long actions from crowding out short ones.