Predicting Cable Dynamics with Physical Attention Bias

Predicting Cable Dynamics with Physical Attention Bias. By Avihai Giuili and three others, from Tel Aviv University and Meta AI. Published on the eighth of October.

Learned simulators for deformable linear objects, such as cables, face two demands. They must predict the motion of cables they were never trained on, and they must stay stable over long rollouts. Most of their error occurs where the cable touches itself or touches the floor. The claim is that giving attention a sense of physical distance helps, and that which distance it uses matters.

The reasoning is about geometry. Attention over every pair of cable segments can represent contact between parts that are far apart along the cable, but attention by itself has no notion of geometry. And a cable has two pairwise distances, which agree only while it is straight. One is arc-length distance along the cable, which governs elastic forces. The other is Euclidean distance in space, which governs contact. Once the cable bends or loops back on itself, two segments can be far apart along its length and yet touching in space. A model that only knew one of those distances would be blind to half of the physics.

The authors add a physical attention bias: an additive term on the attention logits, with a learned rate. Then they ask which distance it should use. They compare four variants. No bias, arc length alone, Euclidean distance alone, and both distances placed on disjoint sets of heads. Everything else, the rest of the model and the training protocol, is held fixed. That control is what lets the comparison speak to the question. Any difference between the variants comes from the bias and the distance it encodes, not from changes elsewhere in the setup.

A physical bias improves prediction on unseen cables. The arc-length bias reduces prediction error by fifteen percent and more than halves the drift in segment length. The Euclidean bias alone stays close to plain attention with no bias. Assigning both distances across heads is best or near best on every metric reported.

The largest gain appears under a specific condition: when attention is the only mechanism that connects distant segments. The fifteen percent figure belongs to that setting, not to every configuration. The study is also a controlled comparison of bias choices inside one model and one training protocol, and its generalization test is to cables unseen in training.

The intuition that contact should be the thing to encode, since contact is where the errors are, does not carry the result on its own. The Euclidean bias barely moves. It is the arc-length bias, the distance that describes the cable's own elasticity, that does the most alone, and giving different heads different distances is the safest choice overall. Code and per-run records are released.