Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill ensemble with Proxy-Dirichlet reverse KL (EnD^2) instead of standard ensemble distillation (EnD) | ResNet-50 student · ImageNet-C | ECE: 5.9 -> 4.5 | Helped |
| EnD^2 instead of EnD | ResNet-50 student · ImageNet-R | ECE: 16.1 -> 11.8 | Helped |
| EnD^2 instead of EnD | ResNet-50 student · ImageNet-val | accuracy: 77.0 -> 77.1 | No clear effect |
| EnD^2 instead of EnD | ResNet-50 student · ImageNet-A | ECE: 46.6 -> 42.8 | Helped |
| distill ensemble into a single model with EnD^2 versus training a single model | ResNet-50 · ImageNet-val | accuracy: 75.9 -> 77.1 | Helped |
| EnD^2 versus ensemble of 10 models | ResNet-50 · ImageNet-val | accuracy: 79.0 -> 77.1 | Hurt |
| EnD^2 instead of EnD for OOD detection with total uncertainty | ResNet-50 student · ImageNet-O | ROC AUC (EoE): 48.4 -> 52.0 | Helped |
| EnD^2 instead of EnD for OOD detection | ResNet-50 student · ImageNet-A | ROC AUC (EoE): 87.2 -> 86.8 | No clear effect |
| approximate ensemble with Proxy Dirichlet | ResNet-50 ensemble · ImageNet-O | ROC AUC (RMI): 62.7 -> 62.9 | No clear effect |
| EnD^2 knowledge uncertainty (RMI) versus ensemble | ResNet-50 · ImageNet-O | ROC AUC (RMI): 62.7 -> 53.2 | Hurt |
| EnD^2 instead of EnD | Transformer-big student · WMT'17 En-De | BLEU: 29.4 -> 29.5 | No clear effect |
| EnD^2 instead of EnD for OOD detection on French input | Transformer-big student · WMT'17 En-De French | ROC AUC (RMI vs EoE): 15.6 -> 15.9 | Mixed |
Closest papers by meaning
- Ensemble Distribution Distillation via Flow Matching (ICML 2025 · similarity 0.86)
- How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation (AAAI 2024 · similarity 0.85)
- Structure-informed Risk Minimization for Robust Ensemble Learning (ICML 2025 · similarity 0.83)
- Knowledge Distillation of Uncertainty using Deep Latent Factor Model (NeurIPS 2025 · similarity 0.83)
- Out-of-Distribution Detection via Deep Multi-Comprehension Ensemble (ICML 2024 · similarity 0.83)
- No Free Lunch from Random Feature Ensembles: Scaling Laws and Near-Optimality Conditions (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.