WeVisOne-VL: Unified Visual Perception from Objects to Pixels

WeVisOne-VL. Unified Visual Perception from Objects to Pixels. By Tianming Liang and seven others, from Tencent. Published on the twenty-second of October. Its code repository is worked out: about fifty-five source files, with a readme and a licence.

Vision-language models can describe a picture in broad strokes, but the authors' starting point is that they remain unreliable at fine-grained visual perception. Knowing roughly what is in an image is not the same as pointing at it precisely, counting it correctly, or tracing its exact outline down to the pixel.

WeVisOne-VL, short for WeChat Vision All-in-One, is a single VLM meant to close that distance. It unifies object-level and pixel-level perception through structured next-token prediction, so one model handles object detection, referring localization, counting, OCR, segmentation and matting. The point is granularity on demand. A user can ask for a location as a point, a box, a mask, or an alpha matte, and the same model answers at whichever level was requested.

Most of that runs through the language model's ordinary token stream. For the finest outputs, the architecture adds a lightweight pixel decoder, connected to the VLM through a special object token, which generates instance-level pixel outputs. Training proceeds in two phases: a three-stage curriculum of supervised fine-tuning, followed by reinforcement learning on a mixture of tasks.

The report evaluates the model on more than thirty benchmarks, and describes leading performance on both object-level and pixel-level tasks. Alongside that, it highlights robust rejection when a query asks about an object that is not in the image: the model declines rather than inventing a location for something absent.

The authors are releasing WeVisOne-VL to the open-source community. Their stated aim is to move these models from broad visual understanding toward perception that is reliable enough for real-world deployment, and the bet here is that one model with one output format can carry every granularity at once.

WeVisOne-VL. Unified Visual Perception from Objects to Pixels. By Tianming Liang and seven others, from Tencent. Published on the twenty-second of October. This is the long version: the whole paper rather than the abstract.

Ask a general vision-language model to find every car in a crowded street and three things go wrong. It hallucinates objects that are not there, it misreads what you meant when the description is unfamiliar, and it skips the small, non-salient instances, so recall is incomplete. On the dense detection benchmark Dense200, the four-billion-parameter Qwen3-VL scores a mean F1 of about fifteen. And when you want pixels rather than boxes, the usual answer is to bolt on a separate large segmentation model. This report, from WeChat Vision at Tencent, builds one model that goes from points and boxes all the way to masks and alpha mattes, and that can say: nothing here matches.

Two ideas come first. The first is the output format. Every task is written as structured next-token prediction. Coordinates are normalised and spelled with dedicated tokens running from zero to one thousand, so a box costs four tokens. Detection, referring, pointing, counting and text reading all share that format, and if the queried target is absent, the coordinate field is filled with the word None. The second idea is how such a model is scored. Average precision needs calibrated confidence scores, which a model that simply writes boxes does not produce. So the paper reports F1, the balance of precision and recall, at an overlap of one half, at a near-perfect overlap of nought point nine five, and averaged over the thresholds in between.

The bridge to pixels is a special token. For segmentation and matting, the model emits an OBJ token immediately after each box, and that token's hidden state becomes a representation of the instance. A lightweight decoder of about nine point six million parameters takes three inputs: that hidden state, the predicted box, and features from several intermediate layers of the shared vision encoder. Following Deformable DETR, but with queries built from the language model's output instead of learned embeddings, six deformable blocks let each instance gather features around its box. The query then generates its own dynamic kernel, which is applied to a pixel map fused from the deep features and a shallow network run on the raw image. For matting, three refinement stages upsample and correct the mask, modulated by the instance query, before a sigmoid yields the alpha matte.

Training is a curriculum. Stage one freezes the vision encoder and teaches the new tokens on ninety million examples, mostly capped at fifty boxes per category, because early training on very dense samples made the model prone to repetitive output collapse. Stage two unfreezes everything, raises the limit to one thousand nine hundred and sixty visual tokens and five hundred boxes per response. Stage three is subtle. The decoder is first warmed up with the model on two hundred and fifty million segmentation samples; then only the decoder and the OBJ embedding are kept and attached to the untouched stage-two checkpoint, which is trained with token and pixel steps alternating one to one. Matting comes last, with the whole model frozen except the decoder, on alpha labels the data engine produced with its own trained matting annotator. Finally, reinforcement learning. Each candidate prompt is scored by sampling responses: the score is highest when the model succeeds sometimes and fails sometimes. About a hundred and forty thousand such prompts are kept. The reward cubes each matched box's overlap, so precision near a perfect fit counts most, and divides by the number of true plus predicted objects, which penalises both misses and duplicates. A negative scores one only if the model predicts nothing. Masks themselves are never rewarded.

Across over thirty benchmarks the four-billion model leads most columns. The dense results carry the thesis about recall: on Dense200 its mean F1 is seventy-one point three, against fifty-eight point seven for LocateAnything, the strongest comparable grounding model. Counting is exact match, and on the MixCount test split it reaches forty point four percent against fifteen point two. The comparison that tests the rejection claim is starker. On RefCOCOg with absent targets, Rex-Omni and LocateAnything reject well under one percent of the time; the four-billion model rejects sixty-seven point four percent. On Ref-ZOM, where the segmentation baselines tend to output a mask regardless, its rejection rate is eighty-six against at best forty-one point nine. On matting prompted by a category name, its mean absolute error is less than a quarter of the strongest pipeline's.

The paper has a limitations section, and it names three trade-offs. Limited general-purpose training data may compromise general capabilities relative to the base model. The model answers directly, without chain-of-thought, which may limit complex reasoning. And it is single-turn, with no agent loop. It adds that benchmark prompts are clear while real users are vague, so intent reasoning remains a challenge. Beyond that, no general-capability benchmark is reported, so the first trade-off is stated but not measured. Each number is a single greedy run without error bars. Two headline benchmarks, WeRef-Bench and WeRefMat, are in-house. And precision has a ceiling: at the strictest overlap on COCO, Grounding DINO still scores higher.

The idea to carry away is that one decoder token can carry an object from language to pixels. The model says what and where in tokens it already writes, then hands a single hidden state and a box to a decoder of about nine point six million parameters. Around it sits the part the abstract undersells: a data engine that builds negatives carefully, and a reward that pays for saying nothing when nothing is there.