Research map / Label-efficient and robust learning / Recognition and long-tail
Information Dropout: Learning Optimal Representations Through Noisy Computation
Alessandro Achille, Stefano Soatto
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Gaussian latent with Information Dropout layer in a VAE | VAE · VAE test set, k=1 | variational lower bound: -98.8 -> -100.0 | No clear effect |
| replace Gaussian latent with Information Dropout layer in a VAE | VAE · VAE test set, k=3 | variational lower bound: -98.7 -> -99.1 | No clear effect |
| use Information Dropout instead of binary dropout | fully connected net on MNIST and All-CNN-32 on CIFAR-10 · MNIST / CIFAR-10 | classification error | Improved |
| use adaptive noise instead of constant log-normal noise with same variance (ablation) | Information Dropout · MNIST | classification error | Improved |
| increase beta (ablation) | Information Dropout, All-CNN-32 · Occluded CIFAR | main task test error | Improved |
| increase beta too far (ablation) | Information Dropout, All-CNN-32 · CIFAR-10 (25% filters) | test error | Hurt |
Closest papers by meaning
- Dropout with Expectation-linear Regularization (ICLR 2017 · similarity 0.91)
- Dropout as a Structured Shrinkage Prior (ICML 2019 · similarity 0.90)
- Variational Dropout Sparsifies Deep Neural Networks (ICML 2017 · similarity 0.90)
- Demystifying Dropout (ICML 2019 · similarity 0.89)
- On the Regularization Properties of Structured Dropout (CVPR 2020 · similarity 0.89)
- Stochastic Modified Equations and Dynamics of Dropout Algorithm (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
