PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
Wenjun Hou, Kaishuai Xu
Independent Researcher
Published October 24, 2026 · on the feed October 29, 2026 · ▲ 4 · arXiv 2610.33645
The abstract
GUI grounding requires locating small controls in cluttered screenshots and identifying which control a natural-language instruction refers to. Cropping and enlargement can expose local detail, but the resulting image may still leave ambiguity about the intended target. We propose PreView, an auxiliary preprocessor that improves the inputs to an existing grounding model without further training that model. PreView prepares two complementary types of input features: task-relevant views for localization and candidate-specific cues for target understanding. A trained lightweight auxiliary model predicts and refines a center, crops the screenshot into a local view, and enlarges that view for zoomed-in grounding. It also prepares a visual cue (i.e., a red-circle marker with a location hint) and a textual cue describing the candidate's identity or function. The frozen grounding model uses the prepared inputs to predict the final click. Across five benchmarks, PreView improves the average grounding accuracy of Qwen3-VL-2B, OS-Atlas-4B, and Qwen3-VL-4B without further training these grounding models. Extensive analyses examine the contributions of views and cues, center refinement, and visual input budgets.
The Brief
1:50A short podcast summary of the paper. Written and voiced by AI.
Transcript
PreView. Enhancing GUI Grounding with Preprocessed Views and Cues. By Wenjun Hou and one other, working independently. Published on the twenty-fourth of October.
GUI grounding asks a model to do two things at once. It has to locate small controls in cluttered screenshots, and it has to work out which control a natural-language instruction refers to. Cropping and enlarging the screenshot can expose local detail, which helps with the first. But the enlarged image may still leave the intended target ambiguous, which leaves the second unsolved.
PreView is an auxiliary preprocessor. Instead of changing the grounding model, it improves what that model is given, and the grounding model itself receives no further training. The inputs it prepares are of two complementary kinds: task-relevant views, aimed at localization, and candidate-specific cues, aimed at understanding which target is meant.
A lightweight auxiliary model, which is trained, predicts a center point and then refines it. Around that center it crops the screenshot into a local view and enlarges the view, so grounding can happen zoomed in. It also prepares two cues. The visual cue is a red-circle marker carrying a location hint. The textual cue describes the candidate's identity or function. The grounding model stays frozen, takes these prepared inputs, and predicts the final click.
Across five benchmarks, PreView raises the average grounding accuracy of three grounding models: Qwen3-VL-2B, OS-Atlas-4B, and Qwen3-VL-4B. None of them is trained further. Extensive analyses then look at how much the views and the cues each contribute, what center refinement adds, and how visual input budgets affect the outcome.
A useful way to hear this is as a division of labour. A small trained helper does the looking around: where to zoom, which candidate to circle, what to say about it. The larger grounding model is left to do what it already does, now with a closer view and a hint about what it is looking at.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free