CellTrack-Bench: Microscopic Cell Tracking Benchmark for Vision-Language Models

CellTrack-Bench. Microscopic Cell Tracking Benchmark for Vision-Language Models. By Ilan Theodoro and two others, from Biohub. Published on the twenty-second of October. Its code repository is public and real: about thirty source files.

Large language models perform outstandingly on text-first tasks such as coding, math and writing. Recent training efforts reach further, toward scientific problems that live in highly specialized domains beyond text, such as protein structure and biological images. In those domains, how general-purpose accuracy compares with specialized algorithms is still an open question. Cell tracking is one such domain, and it demands spatiotemporal awareness of how cells behave over time.

CellTrack-Bench assesses vision-language models on cell-tracking-related performance, and it does so against specialized trackers rather than in isolation. The unit being counted is the cell-linking decision. The comparison is the point: not whether a general model can track cells at all, but how it stands next to the algorithms built for the job.

The full benchmark draws on fifteen standard cell-tracking datasets, spanning four imaging modalities. It comprises one thousand five hundred and thirty-seven problems, which correspond to nine thousand and forty-one cell-linking decisions. A hard subset, requiring greater reasoning effort, was selected by identifying failures shared across distinct algorithmic trackers: the cases where trackers built in different ways all go wrong. Code, the stored model outputs, and a pipeline that regenerates the benchmark from the source datasets are released.

With the dataset in hand, the authors do three things. They evaluate different environment harnesses for the models. They benchmark nine frontier closed-source models and thirteen open-weight models. And they provide a detailed analysis of those models' failure modes. The abstract does not report the scores themselves, so how the models fare against the specialized trackers is left to the paper.

The benchmark gives a measurable form to a question that had been left hanging: whether general-purpose models can stand in for specialized tools in a specialized science. Its hard subset is the sharpest edge, because it is assembled from exactly the cases that already defeat the specialists.