Research map / Optimization / Optimizers
Randomized Block-Diagonal Preconditioning for Parallel Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Randomly repartition coordinates across blocks at each iteration instead of using a static partition | Block-diagonal preconditioned optimization · Logistic regression; URL dataset | sample complexity: several orders of magnitude | Helped |
| Randomly repartition coordinates across blocks at each iteration instead of using a static partition | Block-diagonal preconditioned optimization · Logistic regression; Gisette | sample complexity: around 30× | Helped |
| Randomly repartition coordinates across blocks at each iteration instead of using a static partition | Block-diagonal preconditioned optimization · Logistic regression; RCV1 | sample complexity: 2× | Helped |
| Randomly repartition coordinates across blocks at each iteration instead of using a static partition | CoCoA, ADN, and LS · Logistic regression on three datasets | convergence | Helped |
| Use repartitioning rather than static partitioning | Block-diagonal preconditioned optimization · Uniform-correlation quadratic problems | smallest eigenvalue of the convergence matrix: moves the smallest eigenvalue by a factor of \frac{K-1}{n-1}\approx \frac 1 {n_k} closer to 1 | Helped |
| Use repartitioning rather than static partitioning | Block-diagonal preconditioned optimization · Separable-data toy example | smallest eigenvalue of the convergence matrix: \lambda_{\min}(\Lambda_{\mathcal{P}_3}) = 1; \lambda_{\min}(\mathbb{E}[\Lambda_{\mathcal{P}}]) = \frac 1 3 +\frac 2 3 (1 | Mixed |
| Use repartitioning rather than static partitioning | Ridge regression with real datasets · Gisette, Mushroom, and Covtype | smallest eigenvalue of the convergence matrix | Helped |
| Add line search while preserving the static-versus-dynamic partitioning comparison (ablation) | Block-diagonal preconditioned optimization · Synthetic separable, uniform-correlation, and random data | relative convergence behavior | No clear effect |
Closest papers by meaning
- Block Low-Rank Preconditioner with Shared Basis for Stochastic Optimization (NeurIPS 2023 · similarity 0.90)
- Preconditioning Kernel Matrices (ICML 2016 · similarity 0.89)
- Structured Preconditioners in Adaptive Optimization: A Unified Analysis (ICML 2025 · similarity 0.86)
- Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.85)
- On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning (ICML 2025 · similarity 0.85)
- Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.