Research map / Architectures and efficiency / Pruning and efficient tuning
Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Tune regularization hyperparameters during training using validation-set gradients | Neural network models · MNIST, SVHN, CIFAR-10 | test performance | Helped |
| Use hyperparameters found by T1-T2 as fixed values in a fresh training run | Neural network models · MNIST, SVHN, CIFAR-10 | test classification error | Mixed |
| Tune hyperparameters against validation performance | T1-T2 · MNIST, SVHN, CIFAR-10 | test classification error | No clear effect |
| Reduce hyperparameter update frequency to once per 10 elementary parameter updates | T1-T2 with additive noise hyperparameters · Neural network training | computational overhead: about only 30\% that of backpropagation | Helped |
Closest papers by meaning
- Hyperparameter optimization with approximate gradient (ICML 2016 · similarity 0.90)
- On the Parameterization of Second-Order Optimization Effective towards the Infinite Width (ICLR 2024 · similarity 0.87)
- Cross-regularization: Adaptive Model Complexity through Validation Gradients (ICML 2025 · similarity 0.87)
- On the Provable Separation of Scales in Maximal Update Parameterization (ICML 2025 · similarity 0.87)
- Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function (NeurIPS 2025 · similarity 0.86)
- Sheared Backpropagation for Fine-tuning Foundation Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.