Research map / Learning theory and algorithms / Matrix and tensor methods
Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DHSQR, a communication-efficient distributed high-dimensional sparse linear quantile regression method that double-smooths the check loss into a lasso-penalized least squares problem, with theory on l2 convergence and beta-min support recovery.
Problem
Distributed estimation and support recovery for high-dimensional linear quantile regression, where the non-smooth check loss hampers computation and theory in distributed settings, and prior distributed QR methods assume error independent of covariates.
Approach
Kernel-smooth the check loss via a pseudo covariate and pseudo response to reduce quantile regression to lasso-penalized least squares; use a double-smoothing scheme (global bandwidth h, local bandwidth b) so that only p+1 gradient vectors, not the (p+1)x(p+1) Hessian, need communicating, and iterate a Newton-type update allowing covariate-dependent errors.
Key results
- DHSQR's l2 error after a few iterations is close to the pooled estimator and beats Avg-DC and DPQR in all reported settings (N=20000, n=500, p=500).
- Avg-DC fails in support recovery because averaged local estimates are dense, while DHSQR, Pooled DHSQR, DPQR and DREL reach F1 near 1.
- Theory: after a constant number of iterations (logarithmic in m), DHSQR attains the oracle rate sqrt(s log N / N) and the beta-min condition matches the single-machine rate.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DHSQR instead of Avg-DC and DPQR after a few iterations | Distributed quantile regression · Synthetic homoscedastic and heteroscedastic models with heavy-tailed noise | ℓ2-error | Helped |
| Use DHSQR instead of Avg-DC | Distributed quantile regression · Synthetic models with normal errors; varying total and local sample sizes | ℓ2-error | Helped |
| Use DHSQR instead of DREL | Distributed quantile regression · Synthetic heteroscedastic model | ℓ2-error | Helped |
| Use DHSQR instead of DPQR | Distributed quantile regression · Synthetic heavy-tailed noise settings | ℓ2-error | Helped |
| Ignore dependence between errors and covariates | DREL · Synthetic heteroscedastic model | convergence | Hurt |
| Use DHSQR instead of Avg-DC | Distributed quantile regression · Synthetic support recovery settings | F1-score | Helped |
| Average local quantile-regression estimates (Avg-DC) | Avg-DC · Synthetic support recovery settings, especially large N and small n | F1-score | Hurt |
| Increase the number of DHSQR iterations | DHSQR · Synthetic homoscedastic and heteroscedastic models with heavy-tailed noise | ℓ2-error | Helped |
| Increase total or local sample size | DHSQR, DPQR, and DREL · Synthetic models with normal errors | ℓ2-error | Helped |
Surprising findings
- DREL becomes unstable and fails to converge in the heteroscedastic error case, since it ignores dependence between error and covariates.
- DPQR performs poorly under heavy-tailed noise.
Limitations
- Theory requires strong conditions: irrepresentable condition, sub-Gaussian covariates, sparsity s=O(n^r) with r<1/3, and a consistent initial estimator.
- Experiments are limited to simulations with p=500 and a single real-data application described only briefly.
- Assumes randomly and evenly distributed samples across machines (relaxed only partially in a remark).
Experimental setup
Datasets: synthetic Gaussian/t3/Cauchy linear models real data application (unspecified in text)
Benchmarks: synthetic linear quantile regression simulations
Metrics: l2 error precision recall F1-score
Baselines: Avg-DC Pooled DHSQR DREL DPQR
Builds on: communication-efficient surrogate likelihood Newton-type distributed kernel-smoothed quantile regression conquer PSSsp
Closest papers by meaning
- High-Dimensional Structured Quantile Regression (ICML 2017 · similarity 0.90)
- Distributed Learning of Conditional Quantiles in the Reproducing Kernel Hilbert Space (NeurIPS 2022 · similarity 0.90)
- One-shot Distributed Ridge Regression in High Dimensions (ICML 2020 · similarity 0.86)
- One-Shot Weighted Ensemble Estimation for Federated Quantile Regression: Optimal Statistical Guarantees under Heterogeneous Structured Data (ICML 2026 · similarity 0.86)
- Nearly Linear Row Sampling Algorithm for Quantile Regression (ICML 2020 · similarity 0.86)
- Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.