Research map / Probabilistic and scientific modeling / Bayesian optimization
Modeling the Machine Learning Multiverse
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Vary learning rate across the explored range | SGD with momentum versus Adam · VGG-16 on CIFAR-10 | relative test accuracy | Mixed |
| Tune Adam epsilon | Adam versus SGD with momentum · VGG-16 on CIFAR-10 | relative test accuracy: -0.004±0.0002 main effect; 0.008±0.031 total effect | No clear effect |
| Test an interaction between learning rate and epsilon | SGD with momentum versus Adam · VGG-16 on CIFAR-10 | interaction evidence: K = 2635 | No clear effect |
| Scale batch size up with learning rate while keeping both within the plateau | Neural network training · CIFAR-10 and CIFAR-100 | test accuracy | Helped |
| Increase learning rate or batch size beyond the plateau | Neural network training · CIFAR-10 and CIFAR-100 | test accuracy | Hurt |
| Model learning rate and batch size jointly rather than as independent effects | Large-batch training analysis · CIFAR-10, CIFAR-100, Tiny ImageNet | interaction evidence: K = 0.58 | Helped |
| Exclude Tiny ImageNet trials (ablation) | Large-batch training analysis · CIFAR-10 and CIFAR-100 | interaction evidence: K = 0.34 | No clear effect |
| Use a threshold batch size beyond which learning rate is no longer corrective | Large-batch training · Batch sizes up to 2^13 | generalization error | Hurt |
| Use a general large-batch generalization-gap claim without accounting for learning rate | Large-batch training · CIFAR-10 and CIFAR-100 | generalization error | No clear effect |
| Train on Tiny ImageNet in the explored configurations | AlexNet, VGG, and ResNet multiverse · Tiny ImageNet | test accuracy: 0.01 | Hurt |
| Explore the whole hyperparameter space with IVR instead of optimizing with UCB | SVM classifier · Wisconsin Breast Cancer Dataset | understanding of test-accuracy landscape | Helped |
Closest papers by meaning
- Mapping the Multiverse of Latent Representations (ICML 2024 · similarity 0.90)
- Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation (NeurIPS 2025 · similarity 0.85)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.83)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.82)
- LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language (NeurIPS 2024 · similarity 0.82)
- Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.