Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
Meng'en Qin, Yinchen Liu, Mingxuan Cui, Youlu Xing
Shenzhen University of Advanced Technology · University of Electronic Science and Technology of China · Shandong University
Published September 17, 2026 · on the feed September 21, 2026 · ▲ 5 · arXiv 2609.19122
The abstract
Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit mechanism for suppressing redundant components while preserving signal content, but its sparsity coefficient is typically fixed and manually selected. We propose a training-adaptive convolutional sparse coding framework for robust visual signal representation. Specifically, we unfold the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) and treat the sparsity coefficient as a differentiable variable jointly learned with the network parameters. From the information bottleneck perspective, this coefficient controls the trade-off between information retention and compression: the sparsity term promotes compact representations, while the reconstruction term together with task loss preserves task-relevant signal content. We further introduce a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed. Experiments on CIFAR and ImageNet demonstrate competitive clean-data recognition and greatly improved robustness under different input perturbations.
The Brief
2:05A short podcast summary of the paper. Written and voiced by AI.
Transcript
Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation. By Meng'en Qin and three others, from the Shenzhen University of Advanced Technology, the University of Electronic Science and Technology of China, and one other institution. Published on the seventeenth of September. Its code repository is a placeholder: a readme, and nothing to run yet.
Robust downstream prediction from visual signals needs representations that are compact yet sufficient. Convolutional sparse coding offers an explicit mechanism for this, suppressing redundant components while preserving the content of the signal. Its weak spot is the sparsity coefficient, which is usually fixed and chosen by hand. That single number decides how aggressively the representation is compressed, and yet it is set before training and left alone.
The proposal is a convolutional sparse coding framework that adapts during training. The sparsity coefficient becomes a differentiable variable, learned jointly with the network's parameters, instead of a setting someone picks in advance. The network and the amount of compression it works under are optimized together.
Concretely, the sparse coding optimization is unfolded with the Fast Iterative Shrinkage-Thresholding Algorithm, FISTA. The information bottleneck gives the coefficient its meaning: it controls the trade-off between retaining information and compressing it. The sparsity term pushes toward compact representations, while the reconstruction term, together with the task loss, preserves the signal content the task needs. There is also a label-free post- training strategy. With the main network's parameters held fixed, it adjusts the strength of compression for corrupted inputs.
On CIFAR and ImageNet, the framework shows competitive recognition on clean data, and greatly improved robustness under different input perturbations. On clean inputs it keeps pace; under perturbation is where the improvement shows.
A number that used to be tuned by hand is now learned, and can be adjusted again, without labels, once inputs arrive corrupted. The information bottleneck is what names the trade that number was making all along.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every network compresses its input somehow. Here the dial is fixed by hand, and noisy images expose it.
You are building an image classifier that must stay reliable when its inputs get noisy. Its first layers rebuild each image from a few active filter responses, which squeezes out redundant detail.
The road ahead
- 1 · The problem ~7 min A classifier's early layers squeeze out redundant detail, but one hand-picked number fixes how hard they squeeze, everywhere and forever.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Make the sparsity coefficient a learned compression dial, trained with the network, then re-tuned alone on noisy inputs without labels.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Unroll FISTA into layers, make lambda differentiable and learn it with the network, then freeze the network and re-tune lambda on corrupted inputs.
- 6 · The experiment ~6 min You’ll guess: With a learned, adaptable sparsity coefficient, where did the framework show its improvement: on clean images or under input perturbations?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.