Research map / Architectures and efficiency / RNNs and normalization
PEP: Parameter Ensembling by Perturbation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Gaussian parameter perturbation ensembling | ImageNet pretrained networks · ImageNet (ILSVRC2012) | NLL: 1.030 -> 0.997; 1.035 -> 0.940; 0.994 -> 0.950; 1.084 -> 1.023; 1.199 -> 1.164; 1.176 -> 1.165 | Helped |
| Add Gaussian parameter perturbation ensembling | ImageNet pretrained networks · ImageNet (ILSVRC2012) | Top-1 error %: 25.73 -> 25.13; 25.31 -> 23.74; 22.96 -> 22.26; 26.09 -> 25.18; 29.39 -> 28.83; 28.99 -> 28.75 | Helped |
| Add Gaussian parameter perturbation ensembling | ImageNet pretrained networks · ImageNet (ILSVRC2012) | ECE%: 3.47 -> 2.03; 5.47 -> 2.35; 1.80 -> 2.46; 5.08 -> 2.94; 2.52 -> 1.64; 4.77 -> 4.48 | Mixed |
| Add Gaussian parameter perturbation ensembling | MNIST, Fashion MNIST, and CIFAR models · MNIST (MLP) | NLL: 0.096 $\pm$ 0.01 -> 0.079 $\pm$ 0.01 | Helped |
| Add Gaussian parameter perturbation ensembling | MNIST, Fashion MNIST, and CIFAR models · MNIST (CNN) | NLL: 0.036 $\pm$ 0.00 -> 0.034 $\pm$ 0.00 | Helped |
| Add Gaussian parameter perturbation ensembling | MNIST, Fashion MNIST, and CIFAR models · Fashion MNIST | NLL: 0.360 $\pm$ 0.01 -> 0.275 $\pm$ 0.01 | Helped |
| Add Gaussian parameter perturbation ensembling | MNIST, Fashion MNIST, and CIFAR models · CIFAR-10 | NLL: 1.063 $\pm$ 0.03 -> 0.982 $\pm$ 0.02 | Helped |
| Add Gaussian parameter perturbation ensembling | MNIST, Fashion MNIST, and CIFAR models · CIFAR-100 | NLL: 2.685 $\pm$ 0.03 -> 2.651 $\pm$ 0.03 | Helped |
| Use Deep Ensembles instead of PEP | trained MNIST/CIFAR classifiers · MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100 | NLL: Deep Ensembles achieves the best NLL and classification errors in all the experiments. | Mixed |
| Use uniform instead of Gaussian perturbations (ablation) | PEP on an MNIST MLP · MNIST (MLP) | performance | No clear effect |
| Add PEP parameter perturbation ensembling | WideResNet-28x10 trained on five CIFAR-10 classes · CIFAR-10 out-of-distribution detection | symmetrized KLD: 0.47 -> 0.72 | Helped |
| Increase overtraining | baseline models on CIFAR-10 and MNIST · CIFAR-10 and MNIST | PEP effect (log-likelihood gain) | Helped |
Closest papers by meaning
- Snapshot Ensembles: Train 1, Get M for Free (ICLR 2017 · similarity 0.88)
- Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems (NeurIPS 2024 · similarity 0.87)
- Perturb and Correct: Post-Hoc Ensembles using Affine Redundancy (NeurIPS 2026 · similarity 0.86)
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.85)
- Scaling Exponents Across Parameterizations and Optimizers (ICML 2024 · similarity 0.85)
- Verification of Neural Networks Against Convolutional Perturbations via Parameterised Kernels (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.