SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis

SentZero. An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis. By Hangyul Yoon and three others, from KAIST. Published on the twenty-eighth of September. No code repository is linked to it.

Vision-language pretraining on paired chest X-ray images and radiology reports has shown strong potential for medical image understanding. Yet existing methods often still depend on finetuning for each task, because radiology reports are long, clinically dense, and hard to align with the simple prompts that zero-shot use relies on. Recent sentence-level approaches help, using clinical phrases extracted by large language models, but they largely overlook how radiology discourse itself behaves. Two properties matter. Positive pairs lack diversity, which caps further gains. And clinically equivalent sentences recur across many patients, so contrastive learning ends up pushing apart statements that mean the same thing. Those are false negatives.

SentZero is an enhanced, sentence-centric pretraining framework aimed at zero-shot chest X-ray analysis across many tasks at once. Each of the two problems gets its own answer, and a third change works on the image side.

For diversity, a large language model structures and maps sentences at the abstract level, expanding the set of positive pairs. For false negatives, an additional loss term is added to mitigate them. On the image side, sentence-conditioned residual modulation of the visual embeddings lets visual features adapt to the semantic character of each input sentence. The picture is no longer represented the same way no matter which sentence it is being compared with.

Across diverse downstream tasks and datasets, SentZero improves zero-shot generalization and outperforms earlier multi-task zero-shot methods. The abstract states the direction of these results without giving margins, so the size of the improvement, and which tasks gain most, are left for the paper itself. What the claim does establish is the setting: no task-specific finetuning, several tasks at once, and a comparison against methods that attempt the same thing.

What ties the three changes together is the texture of radiology language: many phrasings of one finding, and one finding written up for many patients. SentZero builds its pretraining around that repetition instead of stumbling over it. More ways to say a finding means more positive pairs, and a loss that mitigates false negatives stops the model from being taught that two patients' identical findings are different.