Learning Heterogeneous Tactile Representation for General Dexterous Manipulation
Bing Han, Yichuan Peng, Haoyu Dong, Yuanmeng Hu, Yongyi Jia, Fangjing Wang, Kecheng Zheng, Yiming Zhang and 11 others
Published November 9, 2026 · on the feed November 10, 2026 · ▲ 99 · arXiv 2611.12806
The abstract
Tactile sensing is an indispensable modality for contact-rich dexterous manipulation, a capability that hints at the ultimate frontiers of embodied intelligence. However, commercially available tactile sensors are still at a relatively early stage of development, with each type of sensor offering irreplaceable strengths alongside its own drawbacks. A key challenge is thus to free tactile understanding from sensor-specific physics, so that embodied models can effectively leverage heterogeneous tactile data. To this end, we develop LingBot-Touch, a tactile foundation model characterized by the following properties. (i) Our model yields a unified representation across different tactile sensors (i.e., piezoresistive, magnetic, and visuotactile sensors), and is easily extensible to new sensor types. (ii) The resulting tactile tokens are plug-and-play: they can be seamlessly integrated into policy models across paradigms, from diffusion policy to vision-language-action (VLA) models such as π_{0.5} and LingBot-VLA, consistently enhancing manipulation dexterity. (iii) Extensive experiments, spanning simulation and real-robot deployments across multiple hardware platforms, demonstrate that our tactile representation substantially benefits complex manipulation tasks while maintaining strong computational efficiency. We believe the findings in this work demonstrate the potential of leveraging cross-sensor data to scale tactile learning, and offer encouraging insights for the design of future hardware and embodied models.
The Brief
2:12A short podcast summary of the paper. Written and voiced by AI.
Transcript
Learning Heterogeneous Tactile Representation for General Dexterous Manipulation. By Bing Han and eighteen others. Published on the ninth of November.
Dexterous manipulation that involves a lot of contact depends on touch. Tactile sensing is an indispensable modality for that kind of work, and the authors see it as pointing toward the far frontiers of embodied intelligence. The hardware, however, is still young. Commercially available tactile sensors are at a relatively early stage of development, and each type offers strengths nothing else replaces alongside drawbacks of its own. No single sensor wins outright. The tactile data a field collects is therefore heterogeneous, each stream shaped by the physics of the sensor that produced it.
The key challenge, as the authors put it, is to free tactile understanding from sensor-specific physics, so that embodied models can make effective use of heterogeneous tactile data. Their answer is LingBot-Touch, a tactile foundation model that yields one unified representation across different kinds of tactile sensor.
Three sensor families are named: piezoresistive, magnetic, and visuotactile. The model covers all three, and it is described as easily extensible to new sensor types as they appear.
What the model produces are tactile tokens, and these are designed to be plug-and-play. They can be integrated into policy models across paradigms, from diffusion policy to vision-language-action models, known as VLA models, such as pi zero point five and LingBot-VLA. Whichever kind of policy receives them, the reported effect is the same: the tokens consistently enhance manipulation dexterity.
The experiments are extensive, spanning simulation and real-robot deployments across multiple hardware platforms. Across them, the tactile representation substantially benefits complex manipulation tasks, and it does so while maintaining strong computational efficiency. The abstract reports these results in qualitative terms, without specific figures.
Behind the model sits a bet on diversity as a route to scale. If data from different sensors can be pooled into one representation, every sensor type in use becomes a source of training data instead of a separate island. The authors present the findings as showing the potential of cross-sensor data to scale tactile learning, and as offering encouraging insights for the design of future hardware and embodied models alike.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Robots manipulate almost entirely by sight. Here is why touch has not caught up.
You want robot hands that feel what they touch. Sight cannot tell them how hard they press, whether a grasp is slipping, or what happens under a finger.
The road ahead
- 1 · The problem ~6 min Robot hands need touch, but every kind of touch sensor speaks its own language, so their data cannot be pooled.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Contact is the same event whatever sensor records it, so one encoder can learn a shared representation once sensor-specific physics is stripped away.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Pool touch data from many sensors, pretrain one encoder on it, and feed its frozen tokens to diffusion or VLA policies.
- 6 · The experiment ~6 min You’ll guess: When the tactile representation was added to robot policies, what happened to computational efficiency?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.