Research map / Architectures and efficiency / RNNs and normalization
Dual Feature Reduction for the Sparse-group Lasso and its Adaptive Variant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dual Feature Reduction (DFR), a two-layer strong screening rule (group then variable) for the sparse-group lasso and adaptive sparse-group lasso, using the epsilon-norm dual representation, with KKT checks to guarantee optimality.
Problem
Sparse-group lasso (SGL) and adaptive SGL (aSGL) are computationally expensive to fit along a regularization path, especially when tuning both lambda and alpha; existing screening rules are either exact but costly (GAP safe) or only single-layer (sparsegl).
Approach
Derives strong screening rules for SGL from the dual norm / epsilon-norm representation, applying group screening followed by variable screening within candidate groups, with KKT checks for safety; extends to aSGL by rewriting the aSGL norm as an epsilon-norm with group-specific gamma_g and epsilon'_g.
Key results
- DFR-aSGL achieves improvement factors of 137.3 (order-2) and 54.0 (order-3) on synthetic interaction data, versus 7.4 and 1.2 for sparsegl.
- On real data, DFR-aSGL improves computational cost by over 600 times on scheetz and adenoma.
- Across all linear-model experiments DFR-SGL had only a single KKT violation, and DFR encountered no failed convergences across real datasets while sparsegl failed at several path points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DFR-aSGL instead of sparsegl | Sparse-group screening on synthetic interaction data · Synthetic interaction data, Order 2 | improvement factor: 7.4\pm 0.9 -> 137.3\pm 12.0 | Helped |
| Use DFR-SGL instead of sparsegl | Sparse-group screening on synthetic interaction data · Synthetic interaction data, Order 2 | improvement factor: 7.4\pm 0.9 -> 44.3 \pm 2.4 | Helped |
| Use DFR-aSGL instead of sparsegl | Sparse-group screening on synthetic interaction data · Synthetic interaction data, Order 3 | improvement factor: 1.2\pm 0.3 -> 54.0\pm 10.7 | Helped |
| Use DFR-SGL instead of sparsegl | Sparse-group screening on synthetic interaction data · Synthetic interaction data, Order 3 | improvement factor: 1.2\pm 0.3 -> 23.6\pm 3.1 | Helped |
| Apply DFR-aSGL rather than DFR-SGL | DFR screening on synthetic interaction data · Synthetic interaction data, Order 2 | improvement factor: 44.3 \pm 2.4 -> 137.3\pm 12.0 | Helped |
| Apply DFR-aSGL rather than DFR-SGL | DFR screening on synthetic interaction data · Synthetic interaction data, Order 3 | improvement factor: 23.6\pm 3.1 -> 54.0\pm 10.7 | Helped |
| Use DFR rather than sparsegl | SGL screening on the adenoma and scheetz datasets · adenoma and scheetz | computational cost improvement: over 600 times | Helped |
Surprising findings
- Heuristic DFR input proportion is similar to exact GAP safe, but GAP safe's safe-region computation cancels out its gain in runtime.
- Without screening, SGL failed to converge on adenoma, scheetz, and tumour.
Limitations
- DFR relies on Lipschitz assumptions that may fail, requiring KKT checks to guard against violations.
- DFR-aSGL showed more KKT violations than DFR-SGL, potentially due to hyperparameter dependence in the Lipschitz assumption.
- DFR carries additional assumptions beyond other strong rules because of the second screening layer.
Experimental setup
Datasets: brca1 scheetz trust-experts adenoma celiac tumour synthetic linear and logistic data
Benchmarks: synthetic data brca1 scheetz trust-experts adenoma celiac tumour
Metrics: improvement factor input proportion KKT violations convergence failures
Baselines: sparsegl GAP safe (dynamic) GAP safe (sequential) no screening
Builds on: strong rule sparse-group framework screening sparsegl GAP safe adaptive sparse-group lasso
Closest papers by meaning
- Fast Sparse Group Lasso (NeurIPS 2019 · similarity 0.88)
- GAP Safe Screening Rules for Sparse-Group Lasso (NeurIPS 2016 · similarity 0.88)
- Bilevel learning of the Group Lasso structure (NeurIPS 2018 · similarity 0.86)
- Chunk Dynamic Updating for Group Lasso with ODEs (AAAI 2022 · similarity 0.86)
- Uncorrelated Group LASSO (AAAI 2016 · similarity 0.85)
- Transformer Learns Optimal Variable Selection in Group-Sparse Classification (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.