Honeycomb: Constant-Size Scene Memory Representation for Video World Models
Jack Wei Lun Shi, Kaichen Zhou, Haoyu Chen, Yufeng Weng, Keane Ong, Ruojin Cai, Hang Hua, Justin K. W. Yeoh and 1 others
Harvard University · National University of Singapore · MIT · MIT-IBM Watson AI Lab
Published September 30, 2026 · on the feed October 2, 2026 · ▲ 5 · arXiv 2609.37690
The abstract
Video world models require persistent scene memory to maintain consistency during long-horizon video generation. Existing spatial memories accumulate RGB observations or latent features, increasing storage requirements as generation proceeds. We introduce Honeycomb, a video world model built on HexMemory, our proposed low-rank representation for storing scene features in a fixed-size memory with a total of six spatial and spatiotemporal planes. A feed-forward writer maps each generated chunk into new plane features. As the spatial coverage or temporal range expands, we warp the previous planes while preserving their dimensions, then fuse them with the new features through confidence-weighted pooling and a learned residual correction. A reader retrieves latents from HexMemory to condition subsequent video generation. The writer processes only observations from the new chunk, avoiding per-scene optimization and repeated processing of the full history. Experiments on WorldScore and RealEstate10K demonstrate strong video generation quality and robust revisit consistency while keeping HexMemory feature storage constant throughout generation. Code and additional visualizations are available on our project page at https://jackswl.github.io/honeycomb/.
The Brief
2:10A short podcast summary of the paper. Written and voiced by AI.
Transcript
Honeycomb. Constant-Size Scene Memory Representation for Video World Models. By Jack Wei Lun Shi and eight others, from Harvard University, the National University of Singapore, and two other institutions. Published on the thirtieth of September. Its code repository is worked out: about eighty-five source files, with a readme and a licence.
A video world model generating over a long horizon needs a persistent memory of the scene to stay consistent, so that a place looks the same when the camera comes back to it. Existing spatial memories keep accumulating, either RGB observations or latent features, and their storage requirements increase as generation goes on. The longer the video, the bigger the memory, with no point at which it stops growing.
Honeycomb is a video world model built on HexMemory, a low-rank representation that stores scene features in a memory of fixed size. That memory is a total of six planes, some spatial and some spatiotemporal. However long generation runs, those six planes are the whole store.
Three pieces operate on it. A feed-forward writer maps each newly generated chunk into new plane features. When the spatial coverage or the temporal range expands, the previous planes are warped while keeping their dimensions, then fused with the new features through confidence-weighted pooling plus a learned residual correction. A reader then retrieves latents from HexMemory to condition the next stretch of video.
The writer only ever processes observations from the new chunk. There is no per-scene optimization, and no repeated pass over the full history. That keeps the work of each write tied to the chunk just generated, not to everything generated so far. The memory changes with every chunk, but its size never does: warping and fusing reshape what the planes hold without adding to them.
Experiments on WorldScore and RealEstate10K show strong video generation quality and robust consistency when the camera revisits a location, while the HexMemory feature storage stays constant from the start of generation to the end.
Growth was the cost of remembering. Honeycomb removes it by fixing the memory's shape in advance and making every update a warp and a fuse into the same six planes, so the scene is kept without the store getting any larger.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is a memory that grows for as long as the camera keeps moving.
You are building a video generator that lets a camera wander through a scene for a long time. When the camera turns back to a place it saw earlier, that place should look the same.
The road ahead
- 1 · The problem ~7 min A video generator that wanders a scene for a long time forgets old places, and today's fixes keep a store that grows without end.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Store the scene in six fixed-size feature planes, write each new chunk in with a quick network, and read latents back to guide the next chunk.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Each round: write the new chunk into plane features, warp the old planes if bounds grow, fuse by confidence, then read latents for the next chunk.
- 6 · The experiment ~6 min You’ll guess: As Honeycomb generated longer and longer videos, what happened to the feature storage its scene memory used?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.