Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable

Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable. By Mengtan Zhang and five others, from Tongji University. Published on the sixteenth of October. Its code repository is worked out: about forty source files, with a readme and a licence.

Self-supervised monocular depth estimation is appealing because it could scale with unlabeled data. In practice, existing methods often degrade when trained on diverse scenes. The study traces this to what the models learn. Under self-supervision, they tend to encode structural biases specific to each dataset rather than scene geometry that transfers, and mixing datasets then produces optimization conflicts across domains. Adding data, which should be the strength of self-supervision, ends up working against it.

MaSS is a framework aimed at making self-supervised depth learning scalable to diverse data. Its central move is to separate structure from context. It introduces a dedicated pathway for structural representation, explicitly decoupled from contextual features. Structure gets its own route through the model, rather than being entangled with everything else an image contains.

That pathway is guided by a new objective: scene-structure discrimination. Its purpose is to learn structural representations that are generalized and layout-aware, and those representations are what the depth decoder works from. The abstract says what the objective is meant to produce, not how it is posed.

MaSS is trained and evaluated on four diverse datasets, both one at a time and jointly. It substantially outperforms prior self-supervised methods, and it is the first to consistently benefit from mixed-domain training, which is the failure the study set out to fix. Zero-shot evaluation on eight additional datasets points to strong generalization, particularly on scenes entirely out of distribution. The source code is public.

The title states the thesis in one line: learning to tell scene structures apart is what makes this kind of depth learning scale. If dataset-specific bias is why more varied data used to hurt, then a pathway that learns transferable structure separately from context is what lets that data help.