Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors

Tactile-JEPA. Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors. By Elizaveta Kovtun and three others, from Sber AI, the Skolkovo Institute of Science and Technology, and two other institutions. Published on the twenty-first of September. Its code repository is worked out: about one hundred and sixty source files, with a readme and a licence.

Touch is an essential sense for robots doing contact-rich, dexterous manipulation, and especially so when vision is occluded. Yet the two senses are treated very differently in practice. Pre-trained image encoders are standard in robot learning pipelines, while tactile encoders are still often trained from scratch on raw, noisy signals, which may limit how expressive they can be. A vision system usually starts from an encoder that has already learned something general. A touch system, much of the time, starts from nothing and has to learn from the noise directly.

Self-supervised pre-training exists for touch, but it has focused mostly on vision-based tactile sensors. Distributed electronic skins have been largely left out. These sensors have a distinctive property: their sensing elements are sparse and irregularly arranged across the surface they cover. That makes simply reusing visual self-supervised methods suboptimal. Those methods were built for readings arranged like an image, and an electronic skin does not arrange its readings that way. The gap, then, is a pre-training method that takes the skin's own layout as its starting point.

Tactile-JEPA is an efficient self-supervised pre-training method that turns that irregular layout from an obstacle into information. It uses the spatial arrangement of the sensors to learn representations that are aware of the sensor topology.

The training task is prediction. Some sensing elements are masked, and the model predicts their embeddings from the unmasked remainder. Which elements get masked is not arbitrary: the sensor connectivity graph guides the spatial masking. The authors' analysis adds a further requirement. Good tactile representations need to capture two things at once, the local details of a contact and the global state of the whole tactile surface. A representation that sees only the fine detail misses the overall picture, and one that sees only the overall picture misses the detail. Tactile-JEPA meets both requirements through dual-scale masking, so that the prediction task itself asks for local and global understanding together.

Testing spans three diverse datasets. They cover magnetic and piezoresistive sensors, different robot embodiments, and both single-sensor and paired-sensor configurations. Against the prior state of the art, Tactile-JEPA reduces force estimation error by six point three percent, and in-hand orientation error by twenty point eight percent. Gains are consistent in other downstream applications as well, including policy learning, where the representation is used to decide what the robot does rather than only to estimate what it feels. Those numbers are not from one sensor on one robot. They hold across two sensing technologies, several embodiments, and setups with one skin or a pair of them.

The authors frame the result as a statement about where the value of touch comes from. How much a robot benefits from tactile sensing, on their reading, depends critically on how well the encoder was pre-trained. Tactile-JEPA is offered as a direct answer to that, one built around the geometry of the skin rather than borrowed from images.