ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation

ENEAS. Embedding-guided Neural Ensemble for Adaptive Segmentation. By Javier del Pino and four others, from SperidLabs. Published on the third of September. Its code repository is worked out: about eighty source files, with a readme and a licence.

Text-promptable segmentation models let you name a thing in words and get back the pixels that belong to it. Even the latest foundation models of this kind, SAM 3 included, still fail in three recognisable ways. The first is temporal hallucination: when an object leaves the field of view, the model does not report that the target is gone. The second is spatial fragmentation: during an extreme close-up, it segments a local texture instead of the whole object. The third is semantic misclassification. The model trusts visual features over ontological reality, so a statue, a painting or a reflection of the target gets segmented as though it were the real thing.

ENEAS is one text-promptable method that works in two modes. In the first it tracks a single, unique instance with precise, high-quality segmentation. In the second it performs open-concept discovery, finding every instance that a text query names, with the results settled by a semantic verification layer. Each mode is aimed at the failures that hurt it most.

For tracking, the starting point is the SeC architecture, described as geometrically robust but previously limited to point interactions. ENEAS adds a text-prompting adapter to it, so it can be driven by words, and relies on its temporal memory. That memory is what lets the target be held through a disappearance without the tracker drifting onto a distractor, and lets the object stay whole even when it fills the entire view.

For discovery, the verification layer works in two stages. High-speed visual embedding matching comes first. Conditional refinement by a VLM follows, and semantic reasoning is invoked only for candidates that remain ambiguous. That is how the layer filters out the ontological errors that visual-only models cannot tell apart, the statue mistaken for the person, while keeping latency low, since the expensive reasoning step runs only where it is needed.

The abstract states the method's purpose more than its numbers. It is designed with 3D reconstruction in mind, a setting where a single misclassified distractor is enough to corrupt the reconstructed asset. In that light, the claim is that ENEAS unlocks high-quality semantic tracking and segmentation across video, across broad libraries, and across collections of data that are unordered in time or in space. With it comes the ability to discriminate true instances from their doppelgangers, things that look alike but are not the same. No benchmark figures are given in the abstract. Code and models are released.

The word that ties the design together is discrimination. A segmentation model that only matches appearance will happily outline a reflection, lose track of an object the moment it walks away, or break a close-up into patches. ENEAS answers each weakness with a different tool: memory for time, a whole-object tracker for scale, and a verification step that brings in reasoning only when looks alone cannot decide. The distinction it cares about is the one in its closing phrase, between an object and something that merely resembles it.