SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

SQS. Bayesian DNN Compression through Sparse Quantized Sub-distributions. By Ziyi Wang and three others, from Purdue University and the University of Texas at El Paso. Published on the seventh of September. Its code repository is worked out: about sixty-five source files, with a readme and a licence.

Deploying a large neural network on a resource-constrained device means compressing it first, and the two standard tools for that are weight pruning and low-bit quantization. Most existing methods reach for one of them at a time. Used individually, each has to stop short: to keep the drop in performance acceptable, the compression rate it reaches is often suboptimal. Pruning removes weights, quantization shrinks the ones that remain, and treating them as separate problems leaves compression on the table.

SQS brings the two together in a single framework that prunes and quantizes simultaneously, and it does so through Bayesian variational learning. The key idea is a pairing of priors over the weights. A spike-and-slab prior induces sparsity, which is the pruning half. The weights that survive are modelled with Gaussian Mixture Models, or GMMs, which is what enables low-bit precision, the quantization half. One probabilistic description of the network carries both kinds of compression at once.

Combining these priors has a cost. An objective that involves spike-and-slab priors together with GMMs is intractable, so it cannot be optimised directly. The authors derive an efficient approximation of it, one that makes effective compression possible with minimal loss of accuracy. On the theory side, they give a consistency result for the proposed variational approach, applied to a deep neural network that is both sparse and quantized.

The experiments compress four quite different models: ResNet, BERT-base, Llama three point two, and Qwen two point five. Across them, the method achieves higher compression rates than a line of existing methods, while its performance drops stay comparable to theirs. The comparison is framed at matched cost: similar degradation, more compression.

What SQS changes is the unit of design. Rather than choosing between cutting weights and coarsening them, it writes both into the prior of one Bayesian model and lets variational learning decide. The spike handles which weights disappear, the mixture handles how the rest are represented, and the approximation is what makes that combination trainable in practice.