Research map / Optimization / Optimizers
GR-Gauge: Cost-efficient Training Configuration By Gauging the Gradient Redundancy
Guanjie Wang, Chen Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GR-guided early termination, config search and state reuse together (vs prior HPO) | GR-Gauge · ResNet18 CIFAR-10 | time to 85% accuracy: saves 63.6% vs BOHB | Improved |
| GR-Gauge vs HPO baselines at fixed budget | GR-Gauge · BERT SQuAD, 20000s | F1 (%): 87.47 vs below 75 | Improved |
| GR-Gauge vs BOHB on small budgets for NeuMF and Llama | GR-Gauge · NeuMF 200s, Llama3 low targets | validation / time to target: NeuMF 200s: 36.18 vs BOHB 56.94 | Mixed |
| add state reuse (ablation) | GR-Gauge · ResNet18, BERT, NeuMF | time to target: no-R 1.17x / 3.39x / 1.27x vs 1x | Improved |
| add GR-guided configuration search (ablation) | GR-Gauge · ResNet18 / BERT / NeuMF | time to target: no-RS 1.46x / 4.45x / 2.16x vs 1x | Improved |
| add GR-guided early termination (ablation) | GR-Gauge · ResNet18 / BERT / NeuMF | time to target: no-RT 1.96x / 7.95x / 2.70x vs 1x | Improved |
| reuse aborted training state instead of restarting from scratch (ablation) | GR-Gauge · ResNet18 final accuracy | accuracy (%): 90.75 -> 89.51 | Hurt |
| reuse aborted training state instead of restarting from scratch (ablation) | GR-Gauge · Llama3 final performance | validation performance (%): 60.54 -> 60.81 | No clear effect |
| check metrics too often (k=1) or too rarely (k=50) (ablation) | GR-Gauge · ResNet/NeuMF/Bert | relative GPU time | Hurt |
| compute GR metrics on sampled parameters and per-device GRS | GR-Gauge · all five models | overhead share of total time: overheads commonly less than 4% | Improved |
Closest papers by meaning
- HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML (NeurIPS 2021 · similarity 0.86)
- Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation (ICLR 2022 · similarity 0.86)
- Frugal Optimization for Cost-related Hyperparameters (AAAI 2021 · similarity 0.85)
- Power Lines: Scaling laws for weight decay and batch size in LLM pre-training (NeurIPS 2025 · similarity 0.85)
- TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.84)
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
