Research map / Optimization / Distributed and compressed
Optimal Shrinkage for Distributed Second-Order Optimization
Fangzhao Zhang, Mert Pilanci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add optimal shrinkage (scaling Hessian by 1/(1-d_λ/n)) to distributed Newton's method | distributed Newton's method · synthetic and real ridge regression | log optimality gap vs communication rounds | Improved |
| add optimal shrinkage to iterative Hessian sketch | IHS · real datasets bodyfat, housing, mpg, triazines | log optimality gap vs Newton steps | Improved |
| use determinantal averaging instead of naive averaging for Hessian inverse | distributed Newton's method · real ridge regression | log optimality gap | Improved |
| replace determinantal averaging with shrinkage estimator for covariance resolvent | covariance resolvent estimation · four LIBSVM datasets | relative spectral norm error | Improved |
| use empirical effective dimension of local Hessian (instead of true effective dimension) in shrinkage | distributed Newton's method · real ridge regression | log optimality gap | No clear effect |
| use the small-regularizer shrinkage coefficient d/n (no effective dimension needed) in place of mdλ,i/n | distributed Newton's method · theory only | bias | No clear effect |
Closest papers by meaning
- Skip the Hessian, Keep the Rates: Globalized Semismooth Newton with Lazy Hessian Updates (NeurIPS 2026 · similarity 0.90)
- Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.88)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.87)
- Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations (ICLR 2025 · similarity 0.87)
- Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing (ICLR 2025 · similarity 0.86)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
