CurveCodec 2: Skeleton-agnostic animation compression with a learned entropy model

CurveCodec 2. Skeleton-agnostic animation compression with a learned entropy model. By Mingyi Shi and three others, from the University of Hong Kong and Adobe Research. Published on the third of October. Its code repository is worked out: about forty-five source files, with a readme and a licence.

Skeletal motion is stored as every joint's transform at every frame, yet most of that is implied by the body rather than by what the motion is about. Compression is one way of asking what a motion still has to say once the body is known. A production codec has to answer that for any skeleton, and with a stated error bound. Any skeleton means the codec cannot be built around one body. A stated bound means a user knows in advance how far a decoded joint may stray.

The authors' earlier codec, CurveCodec, matched the mean error of ACL, the production library of modern game engines, using a learned prior over sparse anchors. It did not match ACL's worst case, though, and it counted its payload as floats rather than bits. Those are the two shortfalls this work sets out from.

This time the starting point is a pair of questions. Where does the redundancy of skeletal motion actually lie? And which part of a codec should a learned model take over? The authors answer with measurements, and the measurements give three answers.

One: at production precision, the largest saving comes from predicting each quantized curve from its own past. The second largest comes from choosing, per joint and in closed loop through the hierarchy, which samples not to code. Two: on the gaps such an encoder leaves, a nearest-neighbour oracle over millions of training samples does no better than linear interpolation, and no learned in-betweener the authors tried paid for itself. Three: what a network does learn is the distribution of the residuals the codec has to send.

CurveCodec 2 is assembled from those answers. Every sub-track is coded as a curve in the log map, quantized in closed loop, and thinned to keys selected by rate-distortion. The residuals are entropy-coded under a small learned model, and its integer inference is bit-exact across platforms. The network sits where the measurements said a network helps, and nowhere else. Two contracts are verified on every decoded clip. Under one, each joint stays within ACL's own worst case, to a stated tolerance. Under the other, each clip stays within ACL's mean error.

The held-out test side is four thousand four hundred and seventy-two clips drawn from thirty-three datasets. Under the worst-case contract, at ACL's default precision of zero point zero one centimetres, CurveCodec 2 needs zero point three seven times ACL's bytes.

Under the mean contract, at zero point one centimetres, it needs zero point two two times. It decodes on a single CPU core. And it transfers, without retraining, to a species that was absent from training. Read against the earlier version, both shortfalls are addressed: the worst case is now a contract checked on every clip, and the payload is counted in bytes.

The finished codec reads as a division of labour drawn from measurement. Prediction from a curve's own past and closed-loop selection of what to skip do the heavy lifting. Linear interpolation fills the gaps as well as anything learned. The network is kept to one job, modelling the residuals, because that is where the measurements found it pays.