Feature Recovery for Object Understanding After Irreversible Fire Damage

Feature Recovery for Object Understanding After Irreversible Fire Damage. By Aditi Tiwari and four others, from the University of Illinois Urbana-Champaign. Published on the tenth of September. No code repository is linked to it.

After a fire, objects often go through irreversible physical changes to their geometry, their material state, and their appearance. Detecting and identifying those remnants matters for locating hazards, reconstructing what was there before the incident, and inventorying losses. This is unlike standard image corruptions, because the degradation changes the physical structure of the object itself, not just the picture of it. A standard corruption sits on top of an intact object. Here the object underneath has changed its shape, its material and its look, and a model has to work out what it once was from what is left.

TRACE is a transformation-aware benchmark for understanding objects after a fire. It defines five tasks aimed at localization and at understanding the object as it was before degradation: detecting degraded objects, recovering and retrieving the pristine state, recovering the original material, generating a description of the pristine object, and reasoning about its function.

The benchmark holds about twenty-one thousand four hundred synthetic scenes grounded in real images, together with paired object-level progressions that run from pristine to degraded. Those progressions span four hundred and ninety-nine object identities, across one hundred and eighty-nine categories. Because each object is paired with its own pristine version, and degradation comes in graded levels of severity, performance can be tracked as the damage gets worse.

Existing models degrade sharply as severity rises. From the least severe level to the most severe, the mAP of RF-DETR falls by seventy-one percent in relative terms, and retrieval recall at one for InternVL3.5 drops from ninety-three point eight five to twenty-eight point one one.

The authors also propose a remedy, the Feature Recovery Module, or FRM. It is a plug-and-play module that maps degraded encoder features to representations aligned with the pristine ones, while the host model stays frozen. Trained only with paired feature supervision, it improves scene-level detection, feature recovery for CLIP and SigLIP2, and all four object-level VLM tasks, with larger gains under more severe degradation.

Averaged across VLM hosts and severity levels, the relative gains are twelve point five percent for retrieval, twenty point one for material recovery, thirteen point two for description generation, and twelve point four for functional reasoning.

The benchmark reframes recognition as recognising what an object used to be. Its numbers show current models losing that ability as damage worsens, and FRM shows that some of it can be recovered from the features alone, without retraining the host.