Research map / Optimization / Optimizers
Learning to Mutate with Hypergradient Guided Population
Zhiqiang Tao, Yaliang Li, Bolin Ding, Ce Zhang, Jingren Zhou, Yun Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add learnable teacher-network mutation (vs random perturbation) (ablation) | HPM · CIFAR10 | validation loss: 0.5724 -> 0.5636 | Improved |
| add learnable teacher-network mutation (vs random perturbation) (ablation) | HPM · PTB | validation perplexity: 73.18 -> 70.49 | Improved |
| remove mutation, use extra hypergradient step (ablation) | HPM · CIFAR10 | validation loss: 0.5636 -> 0.6139 | Hurt |
| remove mutation, use extra hypergradient step (ablation) | HPM · PTB | validation perplexity: 70.49 -> 75.24 | Hurt |
| freeze hyperparameters in hypertraining, only mutate (ablation) | HPM · CIFAR10 | validation loss: 0.5636 -> 0.5944 | Hurt |
| freeze hyperparameters in hypertraining, only mutate (ablation) | HPM · PTB | validation perplexity: 70.49 -> 73.76 | Hurt |
| combine STN with random search instead of PBT exploit/explore (ablation) | STN · CIFAR10 | validation loss: 0.5892 -> 0.5817 | No clear effect |
| replace attention teacher with MLP teacher (ablation) | HPM · CIFAR10 | validation loss: 0.5636 -> 0.5696 | No clear effect |
| replace attention teacher with MLP teacher (ablation) | HPM · PTB | validation perplexity: 70.49 -> 70.91 | No clear effect |
| add hypergradient-guided learnable mutation on top of PBT | PBT · CIFAR10 | validation loss: 0.6253 -> 0.5636 | Improved |
| add hypergradient-guided learnable mutation on top of STN | STN · PTB | validation perplexity: 71.49 -> 70.49 | Improved |
| schedule hyperparameters instead of fixed configuration | hyperparameter schedule methods · CIFAR10 | validation loss: 0.6360 -> 0.5636 | Improved |
Closest papers by meaning
- Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation (ICLR 2022 · similarity 0.87)
- Meta-Surrogate Benchmarking for Hyperparameter Optimization (NeurIPS 2019 · similarity 0.86)
- Guided evolutionary strategies: augmenting random search with surrogate gradients (ICML 2019 · similarity 0.85)
- Understanding the Mechanisms of Fast Hyperparameter Transfer (ICLR 2026 · similarity 0.85)
- Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions (ICLR 2019 · similarity 0.85)
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
