Ambient @ EgoLongQA 2026: Distilling Long-Video perception into a Sub-2B Model
Logesh Kumar Umapathi
TeamAmbient
Published September 10, 2026 · on the feed September 14, 2026 · ▲ 3 · arXiv 2609.07154
The abstract
We describe our entry to the EgoLongQA track of the Wearable-AI Challenge in ECCV 2026, which placed first in the <=2B parameter division with 0.8279 on the held-out test set. Our system is a single 2B vision-language model that answers multiple-choice questions about ten-minute egocentric videos in one greedy forward pass; It is obtained by distilling the junior perception module of a tool-using agentic pipeline, not the agent itself into a small student, using teacher traces filtered to those that answered correctly. it reaches 89% of the accuracy of the large agentic pipeline using 1.1% of its parameters. This raises a 27.1% base model to 81.4% on our held-out questions. The 2B backbone has 2.2132B parameters and therefore over the divisional limit, to make the entry admissable we prune the multilingual embedding table from 248,320 to 143,469 rows, reaching 1.9985B with provably identical logits on retained rows.
The Brief
2:01A short podcast summary of the paper. Written and voiced by AI.
Transcript
Ambient @ EgoLongQA 2026. Distilling Long-Video perception into a Sub-2B Model. By Logesh Kumar Umapathi, from Team Ambient. Published on the tenth of September. Its code repository is public and real: fourteen source files.
The EgoLongQA track of the Wearable-AI Challenge at ECCV 2026 asks multiple-choice questions about ten-minute egocentric videos, and one division caps entries at two billion parameters. A large, tool-using agentic pipeline can handle video that long, but it is far bigger than the cap allows. The question is how much of its ability can be packed into a model small enough to enter.
The team's answer is distillation, aimed with some care. They do not distil the agent itself. They distil the junior perception module of the agentic pipeline into a small student, using teacher traces filtered down to only those that answered correctly. The student is a single two-billion-parameter vision-language model that answers in one greedy forward pass.
One practical obstacle remained. The backbone, nominally two billion, actually has two point two one three two billion parameters, which is over the divisional limit. To make the entry admissible, the team pruned the multilingual embedding table from about two hundred and forty-eight thousand rows to about one hundred and forty-three thousand. That brings the model to one point nine nine eight five billion parameters, with provably identical logits on the rows that were kept.
The entry placed first in its division, scoring zero point eight two seven nine on the held-out test set. The student reaches eighty-nine percent of the large agentic pipeline's accuracy, while using one point one percent of its parameters. On the team's own held-out questions, it raises a base model from twenty-seven point one percent to eighty-one point four percent.
Two choices made the entry possible, one about what to copy and one about what to cut. Copying the perception module rather than the whole agent produced a student that answers in a single pass. Cutting embedding rows, in a way that leaves the kept outputs provably unchanged, brought it under the line.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A big agent can watch ten minutes of video. Your model must be tiny.
You must answer four-way multiple-choice questions about ten-minute videos filmed from a wearer's point of view. Your entry may have at most two billion parameters, vision tower included.
The road ahead
- 1 · The problem ~7 min A big tool-using agent answers long-video questions well, but the contest allows only two billion parameters, and small models alone answer badly.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Copy only the big pipeline's junior first-look module into the small model, training only on its correct, unanimous answers.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Run the junior as teacher, keep its correct traces, train a small student on them, then prune unused vocabulary to fit the cap.
- 6 · The experiment ~6 min You’ll guess: The 2B student answers in one greedy forward pass. What share of the large agentic pipeline's accuracy does it reach?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.