FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videos
Maya Moriya, Sigal Raab, Yael Vinker, Tali Dekel
Weizmann Institute of Science · MIT
Published August 31, 2026 · on the feed September 3, 2026 · ▲ 3 · arXiv 2609.00377
The abstract
We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos. Our framework leverages the reasoning power of a pre-trained Vision-Language Model (VLM) equipped with a suite of specialized tools that enable the agent to simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions. To translate visual content into folding programs, we define a parametric space that consists of the paper's geometry and a set of parametric folding actions. Unlike models that predict static crease patterns, our agent operates sequentially and possesses the ability to re-plan its actions, effectively mitigating the compounding errors inherent in multi-step folding. Our approach takes a step toward closing the gap between human origami knowledge, which is primarily shared through unstructured visual demonstrations, and computational methods, which typically rely on structured, parametric representations such as a crease pattern or an executable parametric plan. We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels. Our results demonstrate that by combining VLM reasoning with a set of specialized tools and physical simulation, we can successfully transform unstructured visual demonstrations into executable, physically plausible folding procedures.
The Brief
2:22A short podcast summary of the paper. Written and voiced by AI.
Transcript
FoldingAgent. Inferring Parametric Origami Procedures from Demonstration Videos. By Maya Moriya and three others, from the Weizmann Institute of Science and MIT. Published on the thirty-first of August. Its code repository is public but slight: two source files.
Origami knowledge lives in two very different places. People mostly share it through unstructured visual demonstrations: someone folds, someone else watches. Computational methods, on the other hand, typically rely on structured, parametric representations, such as a crease pattern or an executable plan with explicit parameters. Between a video of hands folding paper and a program a computer can run, there is a gap, and it is that gap the work sets out to narrow.
FoldingAgent is an agentic framework that infers explicit, parametric folding programs directly from origami demonstration videos. At its core is a pre-trained vision-language model, whose reasoning drives the process. Around it sits a suite of specialized tools. With them the agent can simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions.
To turn what it sees into a program, the framework needs a target language. It defines a parametric space made up of the paper's geometry together with a set of parametric folding actions. Predictions are expressed in that space. The agent also works sequentially, step by step through the fold, and it can re-plan its actions along the way. That sets it apart from models that predict a static crease pattern in one go. Folding is a multi-step process, and errors early on compound. The ability to revise the plan is what mitigates that compounding.
Evaluation runs on PurelandFold, a newly curated benchmark of diverse Pureland origami videos, each with ground-truth geometry and action labels. The abstract reports the outcome qualitatively, without figures. Combining the vision-language model's reasoning with specialized tools and physical simulation successfully transforms unstructured visual demonstrations into folding procedures that are executable and physically plausible.
What the agent produces is not a description of a fold but a procedure that can be run and checked. The tools let the agent simulate its steps and check their physical plausibility, and re-planning keeps one bad step from ruining the rest. The result moves origami from something learned by watching toward something written down in a form a machine can follow.
Paper Quest · 7 chapters · about 38 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Origami lives in videos; computers want programs. Bridging that gap is the quest.
You want a computer to learn origami from the thousands of folding videos people share. Computers want structured plans with explicit parameters, but people teach by folding on camera.
The road ahead
- 1 · The problem ~7 min People teach origami by folding on camera, but a computer needs a structured plan with explicit parameters, and getting from one to the other is hard.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min A vision-language model writes the fold as parametric actions, uses tools to simulate and check each step, and re-plans when an early step goes wrong.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Define a parametric action space, give a vision-language model tools, infer actions one transition at a time, and re-plan from checkpoints when steps fail.
- 6 · Field test ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.