DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence
Xu Huang, Ye Huang, Zijun Liao, Yuwei Niu, Xiaojie Li, Menghan Zhou, De Wen Soh, Xiaotong Li and 1 others
Peking University · Singapore University of Technology and Design
Published September 30, 2026 · on the feed October 1, 2026 · ▲ 37 · arXiv 2609.39222
The abstract
High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff between reconstruction fidelity and generation efficiency: high compression image encoder always increases the learning difficulty of diffusion training, resulting in slow model convergence. Recent representation autoencoders speed up the diffusion training by improving the latent feature's expressive capability by replacing VAE encoders with pretrained semantic encoders, yet they are typically limited to moderate compression and lose pixel-level details necessary for faithful reconstruction. To achieve both high compression and fast diffusion training, we propose DC-SAE, a Decoupled Compact Semantic Autoencoder designed for high-compression image generation with accelerated diffusion model convergence. DC-SAE consists of two key components: (1) a macro-level architecture design that leverages semantic encoders to enable higher compression ratios, and (2) a pixel-level encoder that preserves low-level details, ensuring high-fidelity image reconstruction. We empirically demonstrate that DC-SAE performs strongly on image generation tasks, achieving both compact latent representations and efficient training dynamics. Specifically, on the ImageNet dataset with 512 times 512 resolution, DC-SAE achieves 32times spatial compression, with 29.79 PSNR and 3.37 gFID, substantially outperforming the previous state-of-the-art high-compression tokenizer baselines DC-AE by 13.5% and 54.9% on PSNR and gFID, respectively, maintaining comparable throughput and faster diffusion model training convergence. Beyond class-conditional generation, a 1.6B-parameter DiT using DC-SAE achieves 0.84 on GenEval and 86.007 on DPG-Bench for text-to-image generation at 1024times1024 resolution.
The Brief
2:55A short podcast summary of the paper. Written and voiced by AI.
Transcript
DC-SAE. Deep Compression Semantic Autoencoder for Faster Diffusion Convergence. By Xu Huang and eight others, from Peking University and the Singapore University of Technology and Design. Published on the thirtieth of September. Its code repository is worked out: about sixty source files, with a readme and a licence.
Latent image generators run on tokenizers, and scaling those generators calls for tokenizers that compress hard. Hard compression has a price, though. It sets reconstruction fidelity against generation efficiency: an image encoder that compresses heavily makes diffusion training harder to learn, and the model converges slowly. One recent line of work, representation autoencoders, speeds diffusion training up by swapping the usual VAE encoder for a pretrained semantic encoder, which makes the latent features more expressive. Those approaches tend to stop at moderate compression, however, and they lose the pixel-level detail that a faithful reconstruction depends on. Each side of the tradeoff, in short, has a method that wins it and gives up the other.
DC-SAE sets out to have both: high compression and fast diffusion training. The abstract expands the name as a Decoupled Compact Semantic Autoencoder, built for image generation at high compression with quicker convergence of the diffusion model. The decoupling is between two jobs. One part of the design looks after compression and semantics. A separate part looks after the fine detail.
Concretely, there are two key components. The first is a macro-level architecture design that uses semantic encoders to reach higher compression ratios. The second is a pixel-level encoder that preserves low-level details, so that the reconstructed image stays faithful. The abstract describes the components at that level, and goes no further into how they are joined.
The headline test is ImageNet at five hundred and twelve by five hundred and twelve resolution. There DC-SAE compresses spatially by thirty-two times, and reaches a PSNR of twenty-nine point seven nine with a gFID of three point three seven. Against DC-AE, the previous state-of-the-art tokenizer for high compression, that is better by thirteen point five percent on PSNR and by fifty-four point nine percent on gFID. Throughput stays comparable, and the diffusion model trains to convergence faster.
The authors also go past class-conditional generation to text-to-image. A DiT of one point six billion parameters built on DC-SAE, generating at one thousand and twenty-four pixels square, scores zero point eight four on GenEval and eighty-six point zero zero seven on DPG-Bench.
The tradeoff here was framed as a choice between a latent that is easy to diffuse and one that is easy to decode. DC-SAE declines to choose, giving semantics and pixels each an encoder of their own, and the reported numbers improve on both sides of the ledger at once: sharper reconstructions and quicker training, at a compression ratio well past the moderate level where semantic approaches usually stop.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Compress harder and training gets cheaper, until it doesn't. Here is the tradeoff.
You build the image tokenizer that a diffusion model generates in. Compressing images hard makes each training step cheap, but the diffusion model then needs many more steps to converge.
The road ahead
- 1 · The problem ~7 min Compressing images hard makes each diffusion step cheap, but the generator then converges slowly, and semantic encoders that speed it up lose fine detail.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Split the tokenizer into two branches: a frozen semantic branch for a diffusion-friendly latent, and a pixel branch for detail, fused and trained through one decoder.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Build a 32 times semantic encoder, add a pixel encoder beside it, train the tokenizer jointly, then freeze it and train a DiT on the fused latent.
- 6 · The experiment ~6 min You’ll guess: At 32 times spatial compression on ImageNet, by how much did DC-SAE improve on DC-AE's gFID?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.