Kirin: Animal Motion Generation from In-the-Wild Video
Brian Nlong Zhao, Zhuoyang Pan, James M. Rehg, Jiajun Wu, Shangzhe Wu
University of Illinois Urbana-Champaign · University of Pennsylvania · Stanford University · University of Cambridge
Published September 1, 2026 · on the feed September 3, 2026 · ▲ 3 · arXiv 2609.01823
The abstract
Understanding animal motion is fundamental to modeling animal behavior and biomechanics, yet progress in this area lags far behind human motion research due to the scarcity of high-quality motion data. While human motion can be captured in controlled environments, it is impractical for most animal species, resulting in small, domain-limited datasets that restrict downstream applications such as animation. To address this challenge, we introduce Kirin, a framework that reconstructs motion from video, learns motion priors at scale, and generates realistic motion that can be directly applied to animated assets. Using large collections of in-the-wild animal videos, we reconstruct 3D motion sequences and pair them with captions to create AiM3D, the first large-scale dataset offering aligned video-text-motion tuples for quadruped animals. Building on this dataset, we develop a visual-guided motion generation model that conditions on both text and image to guide the generation of realistic motion across diverse animal species. Finally, by leveraging an off-the-shelf image-to-3D model, we automatically rig and animate 3D meshes using generated motion, producing ready-to-render animated animals. Together, our dataset and framework establish a new foundation for large-scale, text and image conditioned animal motion generation and animation. Project page: https://kirin-ani.github.io/.
The Brief
2:19A short podcast summary of the paper. Written and voiced by AI.
Transcript
Kirin. Animal Motion Generation from In-the-Wild Video. By Brian Nlong Zhao and four others, from the University of Illinois Urbana-Champaign, the University of Pennsylvania, and two other institutions. Published on the first of September. No code repository is linked to it.
Understanding how animals move is fundamental to modelling their behaviour and biomechanics, yet this area lags far behind human motion research, and the reason is data. Human motion can be captured in controlled environments. For most animal species that is impractical. What results are small datasets limited to narrow domains, and those restrict downstream applications such as animation.
Kirin gets around the capture problem by starting from video. It is a framework that reconstructs motion from video, learns motion priors at scale, and generates realistic motion that can be applied directly to animated assets. The raw material is large collections of animal videos filmed in the wild, rather than anything recorded in a controlled setting.
There are three parts. First, 3D motion sequences are reconstructed from those videos and paired with captions. The result is AiM3D, which the authors describe as the first large-scale dataset offering aligned video, text and motion tuples for quadruped animals.
Second, building on that dataset, a visual-guided motion generation model conditions on both text and an image to guide the generation of realistic motion across diverse animal species. Third, an off-the-shelf image-to-3D model is used to automatically rig and animate 3D meshes with the generated motion, so what comes out is an animated animal ready to render.
The abstract reports no numbers. What it reports is the reach of the whole chain: realistic motion generated across diverse species, guided by text and by images, and carried all the way to ready-to-render animated assets. The authors claim that the dataset and the framework together establish a new foundation for large-scale animal motion generation and animation conditioned on text and images.
The move is to treat ordinary footage as the motion-capture stage that most animals were never going to walk onto. Once motion can be reconstructed from video found in the wild, the supply of animal motion data stops being limited to what can be captured in a controlled environment, and a dataset, a generator and an animation pipeline can all be built on top of it.
Paper Quest · 7 chapters · about 37 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Human motion has its datasets. Animals have almost none. That is the wall here.
You want to generate realistic animal motion for animation. For humans, motion capture in a lab has given huge, precise datasets. For most animals, that is impractical, and for wild species, impossible.
The road ahead
- 1 · The problem ~7 min Animal motion models are starved of data, because the lab methods that gave humans huge motion datasets do not work for most animals.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Kirin turns in-the-wild animal videos into captioned 3D motion, generates motion from text plus an image, and puts it on a rigged mesh.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Reconstruct 3D motion from videos, caption it into a dataset, train a text-and-image motion generator, then rig a mesh and animate it.
- 6 · Field test ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.