QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation

QCell. Recombining and Aligning Cell Queries for Overlapping Instance Segmentation. By Yaroslav Prytula and two others, from the University of Tartu and the Ukrainian Catholic University. Published on the twenty-ninth of August. Its code repository is public but slight: six source files.

Segmenting individual cells in microscopy gets hard exactly where the cells overlap. The structures are semi-transparent, so a boundary between two cells is weak rather than sharp, and the visual evidence inside an overlap region belongs to both cells at once. Existing methods attack this locally, through regions of interest or shape priors, which means they never reason about the overlapping objects together.

QCell is a query-based model, and the shift is that a query stands for a whole object rather than a patch of image. That gives the model somewhere to do global reasoning about what a complete cell looks like even where part of it is buried under another one.

Two components carry it. An instance recombination module decomposes and recombines query representations in latent space, which is what lets the model reason about a complete object structure under overlap rather than about the visible fragment. Alongside it, a contrastive query alignment objective does two jobs at once: it learns distinctive features per instance, and it pushes apart the queries of overlapping cells so they do not collapse into each other.

The authors also introduce a new Organoid dataset as a benchmark for overlapping cell segmentation, which matters because the problem is defined by the overlap and existing datasets are not built around it. QCell outperforms state-of-the-art methods across multiple benchmarks, gaining two point two AP and two point seven AJI on ISBI2014.

The recombination idea is the part that travels. Where evidence is genuinely shared between two objects, a model that can only see the visible fragment is being asked the wrong question.

QCell. Recombining and Aligning Cell Queries for Overlapping Instance Segmentation. By Yaroslav Prytula and two others, from the University of Tartu and the Ukrainian Catholic University. Published on the twenty-ninth of August. This is the long version: the whole paper rather than the abstract.

Overlapping objects in microscopy are a different problem from overlapping objects in ordinary photographs, and the difference is the whole reason this paper exists. In a natural image, when one thing occludes another, the hidden region is visually absent. In many microscopy modalities the overlap is semi-transparent, so the hidden cell structure remains partially visible, with weak contrast and mixed visual evidence.

That produces a specific difficulty. An overlapping region contains information from more than one instance at once, giving ambiguous boundaries and strong appearance entanglement between neighbours. Resolving it requires amodal reasoning: inferring the complete extent of an object from a partial observation. Existing approaches attack that through region-level decomposition, shape priors, or multi-stage generative completion. Those work, and they share a structural constraint. They typically operate on local region-of-interest features or depend on upstream predictions, which limits their ability to reason jointly over the full scene. And for overlapping cells specifically, understanding how each instance relates to its neighbours is exactly what determines which evidence belongs to which cell.

The central move is to decompose each instance query into three sub-representations: amodal, visible, and invisible. The complete object, the part you can see, and the part hidden by or shared with a neighbour. Those are then recombined with consistency regularization, so the model is pushed toward a coherent account in which the visible and invisible parts add up to the amodal whole. The authors credit an earlier decompose-and-recombine idea and describe their contribution as bringing that principle to the query level, which removes the dependency on region proposals. The specific defect they are addressing is named plainly: a standard model producing a single mask per instance does not model the relationship between visible and occluded parts at all.

The second component addresses a different failure. Nothing so far stops the queries for two overlapping cells collapsing toward each other, since they are looking at largely the same pixels. So contrastive query learning is added, leveraging denoising queries, which supply stable ground-truth-initialized representations during training. It combines an instance-discriminative loss with a query alignment loss, and the second stated limitation of prior work is what this answers: the training objective previously lacked explicit supervision for learning discriminative instance features in dense overlapping scenes.

The authors also introduce a new Organoid dataset as a benchmark for overlapping cell segmentation, which matters because the difficulty here is defined by the overlap and existing datasets are not constructed around it. The method outperforms state-of-the-art approaches across multiple benchmarks, gaining two point two AP and two point seven AJI on ISBI2014. Reporting both is deliberate: average precision rewards detecting instances, while the aggregated index is sensitive to how well each instance's pixels are assigned, which is where an overlap method should show its work. Code is available, and the new dataset is the part most likely to outlast the method.

This is the shortest paper in the batch and its scope is correspondingly narrow. Everything is cell instance segmentation in microscopy, and the specific property being exploited, semi-transparent overlap where hidden structure stays partially visible, is what distinguishes the setting from natural-image occlusion. So the decomposition into visible and invisible parts is well matched here and would have less to work with where the occluded region carries no signal at all. The gains reported are a couple of points on established benchmarks rather than a change of kind, and no ablation is described in the abstract separating the contribution of recombination from that of contrastive alignment, which given the argument that the two components complement each other is the ablation a reader would most want to see.

The transferable idea is treating a shared region as evidence belonging to two objects at once, rather than as a boundary problem to be settled between them. Once an instance is represented by something that can be split into what is seen and what is inferred, the ambiguity in the overlap becomes something the model can represent rather than something it has to resolve prematurely.