From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings

From Fragmentation to Fusion. A Survey and Roadmap for Omnimodal Embeddings. By Ziyin Zhang and seventeen others, from Ant Group, Shanghai Jiao Tong University, and three other institutions. Published on the thirtieth of October.

Embedding models sit underneath virtually every modern AI system, from search engines to recommendation systems. The mathematics behind them is shared. The research is not. Work on embeddings remains split by modality, with each community developing largely in isolation and seldom exchanging insights. This position paper argues that the split hides something deeper: a convergence already under way.

To reason about that convergence without blurring real differences, the authors propose a framework of three capabilities, each harder than the last. Coverage comes first, then comparability, then compositionality. Matching these are three paths toward unification, each satisfying one capability in turn. Transformation broadens coverage. Alignment enables comparability. Joint embedding aims at compositionality, and it is the terminus the authors advocate. The survey traces how the field has moved along this sequence.

Three forces are named as driving embedding research across modalities toward a common set of principles: contrastive learning, Transformer architectures, and a shift in granularity. The authors are careful not to overstate the unity. Even as the principles converge, challenges specific to individual modalities remain underexplored.

The authors present this as the first unified, cross-cutting treatment of omnimodal embedding research. Its usefulness lies in the ladder it offers. Rather than a single verdict on whether modalities have merged, it gives three rungs, coverage, comparability and compositionality, against which any approach can be placed, and it says plainly which rung it thinks the field should be climbing toward.