Research map / Probabilistic and scientific modeling / Variational and generative sampling
The Description Length of Deep Learning models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use prequential coding instead of variational coding | deep networks · MNIST | codelength: 22.2 -> 4.10 kbits | Helped |
| use prequential coding instead of variational coding | deep networks · CIFAR10 | codelength: 89.0 -> 45.3 kbits | Helped |
| switch among prediction strategies | prequential coding · CIFAR10 | codelength: 45.3 -> 34.6 kbits | Helped |
| switch among prediction strategies | prequential coding · MNIST | codelength: 4.10 -> 4.05 kbits | No clear effect |
| switch among training stopping times within one model | prequential coding · CIFAR10 | codelength: 45.3 -> 34.9 kbits | Helped |
| switch among training stopping times within one model | prequential coding · MNIST | codelength: 4.10 -> 4.05 kbits | No clear effect |
| add data augmentation and batch normalization to the VGG-like model (ablation) | VGGa prequential coding on CIFAR10 · CIFAR10 | codelength | Helped |
| use a learned prequential network rather than uniform encoding | MNIST · MNIST | codelength: 199 -> 4.10 kbits | Helped |
| use a learned prequential network rather than uniform encoding | CIFAR10 · CIFAR10 | codelength: 166 -> 45.3 kbits | Helped |
Closest papers by meaning
- Does Compression Imply Generalization? A Minimum Description Length Perspective (NeurIPS 2026 · similarity 0.90)
- Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters (ICLR 2019 · similarity 0.89)
- Compressing Neural Networks using the Variational Information Bottleneck (ICML 2018 · similarity 0.88)
- Compression-aware Training of Deep Networks (NeurIPS 2017 · similarity 0.88)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.86)
- Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.