Research map / Optimization / Distributed and compressed
Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization
Michal Derezinski, Burak Bartan, Mert Pilanci, Michael W. Mahoney
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| rescale regularization to lambda' = lambda(1 - d_lambda/m) (ablation) | distributed Hessian sketch averaging (surrogate and Gaussian sketch) · regularized least squares, CIFAR-10 and Boston housing | estimation error | Improved |
| rescale regularization for uniform row sampling (ablation) | uniform sampling sketch · regularized least squares | estimation bias | Mixed |
| use surrogate row sampling sketch instead of i.i.d. row sampling with unweighted or determinantal averaging | distributed iterative Hessian sketch · Boston housing, lambda=10, m=50 | estimation error | Improved |
| rescaled regularization in distributed Newton sketch (ablation) | distributed Newton sketch, logistic regression · three UCI binary classification datasets, q=100 | convergence speed | Improved |
Closest papers by meaning
- Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing (ICLR 2025 · similarity 0.88)
- Distributed Least Squares in Small Space via Sketching and Bias Reduction (NeurIPS 2024 · similarity 0.87)
- Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations (ICLR 2025 · similarity 0.86)
- Fundamental Bias in Inverting Random Sampling Matrices with Application to Sub-sampled Newton (ICML 2025 · similarity 0.86)
- Second-Order Convergence in Private Stochastic Non-Convex Optimization (NeurIPS 2025 · similarity 0.86)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
