Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers
Jiongfan Yang, Weichen Fan, Haiwen Diao, Zhongang Cai, Ziwei Liu
Peking University · Nanyang Technological University · SenseTime
Published October 29, 2026 · on the feed October 30, 2026 · ▲ 31 · arXiv 2610.40647
The abstract
Pixel-space generative Transformers typically use one token per image patch, so capturing finer detail requires more tokens and makes global attention increasingly expensive. We observe that finer detail does not necessarily require more global communication: low-frequency structure benefits from global exchange, while high-frequency detail can be modeled locally. Based on this insight, we introduce SpecSlot, a pixel-space rectified-flow Transformer that decouples output detail from the length of the global attention sequence. SpecSlot expands each spatial anchor into multiple slots, each assigned a disjoint band of DCT coefficients; this separates anchor density from spectral subdivision. Only the lowest-frequency slot from each anchor participates in global attention, while all slots interact locally. This allows the model to increase spectral detail without enlarging the global sequence. A spatial head further adds a local correction to the spectral prediction, while the generative state, target, and loss remain in pixel space. On ImageNet-256, SpecSlot achieves a comparable final FID to a FLOPs-matched JiT control while converging faster. On ImageNet-512, it improves FID by 2.84 over the control with 3.9% fewer FLOPs, and comes within only 0.69 FID of JiT-B/16 at 38.2% lower cost. On text-to-image generation, SpecSlot surpasses JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.
The Brief
2:23A short podcast summary of the paper. Written and voiced by AI.
Transcript
Read, Allocate, Route. Role-Coupled Tokenization for Pixel-Space Generative Transformers. By Jiongfan Yang and four others, from Peking University, Nanyang Technological University, and one other institution. Published on the twenty-ninth of October.
Pixel-space generative Transformers typically use one token per image patch. That ties detail to sequence length: capturing finer detail takes more tokens, and more tokens make global attention ever more expensive. The patch grid is doing several jobs at once. It sets how much detail the output can carry, and it sets how many tokens have to talk to every other token. Turning up one turns up the other, and the cost of global attention climbs as the detail does.
The observation that breaks the tie is that finer detail does not necessarily need more global communication. Low-frequency structure benefits from global exchange, while high-frequency detail can be modeled locally. SpecSlot, a pixel-space rectified-flow Transformer, is built on that split, so that output detail is decoupled from the length of the global attention sequence.
Each spatial anchor is expanded into several slots, and each slot is assigned a disjoint band of DCT coefficients. That separates how densely anchors are placed from how finely the spectrum is subdivided. Only the lowest-frequency slot from each anchor takes part in global attention, while all slots interact locally, so spectral detail can grow without lengthening the global sequence. A spatial head adds a local correction to the spectral prediction, and the generative state, the target and the loss all stay in pixel space. The spectral slots are a way of organising tokens, then, not a change of what the model is generating. Adding finer frequency bands adds local work, while the global sequence stays the length set by the anchors.
On ImageNet-256, SpecSlot reaches a final FID comparable to a FLOPs-matched JiT control, and converges faster. On ImageNet-512, it improves FID by two point eight four over that control while using three point nine percent fewer FLOPs.
It also comes within zero point six nine FID of JiT-B/16, at thirty-eight point two percent lower cost. In text-to-image generation, SpecSlot surpasses JiT-B/8 on DPG-Bench by one point zero six, at forty point six percent lower forward cost.
The design rests on routing each kind of information to the communication it needs. Global structure travels through the short global sequence, fine detail stays local, and the model gains resolution without paying for it in global attention.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here, more detail means more tokens talking to everyone.
You are building a Transformer that generates images directly in pixels, one token per image patch. To get finer detail, you shrink the patches.
The road ahead
- 1 · The problem ~7 min In a pixel-space image Transformer, finer detail means smaller patches, more tokens, and a global attention bill that grows with the square of the token count.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Give each anchor several slots that own different DCT frequency bands, and let only the lowest-frequency slot join global attention.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Read anchors, expand them into slots, give each slot a DCT band, route carriers globally and all slots locally, then add a spatial correction.
- 6 · The experiment ~6 min You’ll guess: On ImageNet-256, how did SpecSlot's final FID compare with a FLOPs-matched JiT control?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.