Research map / Probabilistic and scientific modeling / Variational and generative sampling
Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace BB-ELBO with importance-sampling bits-back coding using N particles | VAE and VQ-VAE latent-variable compressors · MNIST and CIFAR-10 test sets | bits/dim: MNIST: 0.236 -> 0.230 (ELBO); 0.236 -> 0.228 (IWAE). CIFAR-10: 4.898 -> 4.857 (ELBO); 4.898 -> 4.810 (IWAE) | Helped |
| Increase the number of importance-sampling particles from 5 to N (ablation) | BB-IS · MNIST and CIFAR-10 test sets | bits/dim: MNIST ELBO: 0.232 -> 0.230; MNIST IWAE: 0.231 -> 0.228; CIFAR-10 ELBO: 4.866 -> 4.857; CIFAR-10 IWAE: 4.827 -> 4.810 | Helped |
| Train the model with the corresponding IWAE bound and match the particle count at compression (ablation) | BB-IS · MNIST and CIFAR-10 test sets | bits/dim: MNIST: 0.230 -> 0.228; CIFAR-10: 4.857 -> 4.810 | Helped |
| Use BB-IS rather than BB-ELBO on transferred, out-of-distribution data | VAE · EMNIST-MNIST and Letters test sets under both training distributions | bits/dim: 0.236 -> 0.228; 0.310 -> 0.280; 0.257 -> 0.244; 0.250 -> 0.239 | Helped |
| Couple particle randomness to reduce the initial-bit cost (ablation) | BB-IS, producing BB-CIS · MNIST and Letters test sets | total bitrate (bits/dim): MNIST: 0.230 -> 0.228; Letters: 0.241 -> 0.239 | Helped |
| Use coupled particles instead of independent particles (ablation) | BB-IS · Synthetic mixture model | net bitrate | No clear effect |
| Use BB-SMC rather than BB-IS with the same particle count | VRNN music compression · MuseData, Nottingham, JSB, and Piano-midi.de test sets | bits/timestep: MuseData: 10.66 -> 9.58; Nottingham: 4.86 -> 4.76; JSB: 12.03 -> 10.92; Piano-midi.de: 11.38 -> 11.20 | Helped |
| Use BB-SMC rather than BB-ELBO | VRNN music compression · MuseData, Nottingham, JSB, and Piano-midi.de test sets | bits/timestep: MuseData: 10.66 -> 9.58; Nottingham: 5.87 -> 4.76; JSB: 12.53 -> 10.92; Piano-midi.de: 11.43 -> 11.20 | Helped |
| Replace importance sampling with sequential Monte Carlo resampling (ablation) | Bits-back coding on a toy HMM · Toy HMM | net bitrate convergence | Helped |
| Apply BitSwap to BB-AIS (ablation) | BB-AIS · Synthetic mixture model | net bitrate convergence to entropy | Hurt |
| Apply amortized-iterative inference to BB-IS (ablation) | BB-IS lossy image compression · EMNIST-MNIST test set | net bitrate savings | Helped |
| Use BB-IS in place of BB-ELBO | Two-layer VAE with hyperprior · EMNIST-MNIST lossy compression | net bitrate savings: more than 15 % | Helped |
Closest papers by meaning
- Practical lossless compression with latent variables using bits back coding (ICLR 2019 · similarity 0.91)
- Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables (ICML 2019 · similarity 0.88)
- Compression with Flows via Local Bits-Back Coding (NeurIPS 2019 · similarity 0.88)
- Beyond Communication Overhead: A Multilevel Monte Carlo Approach for Mitigating Compression Bias in Distributed Learning (ICML 2025 · similarity 0.85)
- Progressive Compression with Universally Quantized Diffusion Models (ICLR 2025 · similarity 0.85)
- Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.