DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation
Can Li, Jie Gu, Zishun Deng, Jingmin Chen, Lei Sun
Nankai University · Rightly Robotics
Published September 17, 2026 · on the feed September 21, 2026 · ▲ 14 · arXiv 2609.18620
The abstract
Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/
The Brief
2:15A short podcast summary of the paper. Written and voiced by AI.
Transcript
DeformSmith. Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation. By Can Li and four others, from Nankai University and Rightly Robotics. Published on the seventeenth of September. Its code repository is a placeholder: a readme, and nothing to run yet.
Building a deformable asset for robot manipulation means specifying its geometry, its appearance, and its physical properties together. For deformable objects this is especially hard. Text and images say little about how an object deforms or responds to contact, yet those responses decide whether the asset is fit for interaction at all. Automated generation therefore has to resolve physical requirements that are coupled to one another, and has to use evidence from interaction to guide how the asset is built and refined.
DeformSmith is a framework for generating deformable assets automatically, from text or from a single image, that are interactive and physically credible. It rests on two things. The first is hierarchical agentic construction, building the asset in stages rather than in one pass. The second is a shared harness grounded in physics, a common place where each stage of the asset is tested. The aim is an asset that is not just a shape with a texture, but one ready to be picked up, pressed, and deformed by a robot in simulation.
With those, the framework builds, tests, and refines progressively: geometry, physical models, material behavior, and robot interaction, continuing until the asset is ready for simulation and manipulation. Robot interaction is what closes the generation loop. Manipulation feedback and replayable interaction data flow back in, so an asset is judged by how it behaves when handled, not only by how it looks.
The results compare DeformSmith with state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni. Its assets show better visual quality and better physical plausibility, and the framework also supports synthesizing data for robotic manipulation of deformable objects. So the output serves twice: as an asset for simulation, and as a source of interaction data for learning to handle soft things.
Text and a picture cannot say how something squashes. DeformSmith's response is to find out by testing, with a robot in the loop, and to keep refining until the asset holds up under the handling it was made for.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Soft things are hard: a photo shows a shape, not a squish. That is the wall.
You need soft objects for a robot simulator: a sponge, a plush toy, a dough ball. Each needs geometry, appearance and physical properties, all specified together, from just a text prompt or one photo.
The road ahead
- 1 · The problem ~7 min Soft objects for a robot simulator need shape, looks and physics set together, yet a text prompt or photo barely hints at how they squash.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Build the soft asset in ordered stages, test every stage in one shared physics loop, and finish by letting a simulated robot handle it.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Four layers run in order, from geometry to physical model to material to robot handling, and the robot runs are recorded as replayable data.
- 6 · The experiment ~6 min You’ll guess: Compared with baselines PhysGen3D, PhysGM and PhysX-Omni, how did DeformSmith's generated assets fare?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.