Research map / Label-efficient and robust learning / Recognition and long-tail
On the Stability of Feature Selection Algorithms
Sarah Nogueira, Konstantinos Sechidis, Gavin Brown
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Choose LASSO lambda by maximizing stability along with likelihood instead of likelihood only | L1-logistic regression · Synthetic data, low redundancy | false positives: FP = 14 -> FP = 0 (FN 19 -> 20) | Improved |
| Choose LASSO lambda by likelihood and stability instead of likelihood only | L1-logistic regression · Synthetic data, high redundancy | false positives: FP = 12 -> FP = 0 (FN 38 -> 39) | Improved |
| Choose lambda by likelihood and stability instead of likelihood only | L1-logistic regression · Synthetic data, medium redundancy | false positives / false negatives: FP = 0, FN = 26 -> FP = 0, FN = 26 | No clear effect |
| Choose lambda by likelihood and stability instead of likelihood only | L1-logistic regression · Synthetic data, no redundancy | false positives / false negatives: FP = 0, FN = 0 -> IDEM | No clear effect |
| Increase LASSO lambda from 0.0051 (likelihood optimum) to 0.0187 (ablation) | L1-logistic regression · Synthetic data, low redundancy | stability: stability up to 0.63 with negligible likelihood loss | Improved |
| Pick lambda to optimize stability as well as likelihood | L1-logistic regression · Synthetic data, all redundancy levels | misclassification error | No clear effect |
| Pick Elastic Net lambda 0.76 instead of likelihood-optimal 0.05 (ablation) | Elastic Net alpha=0.05 · Synthetic data, high redundancy | stability: 0.34 -> 0.98 | Improved |
| Pick Elastic Net lambda 0.76 instead of 0.05 (ablation) | Elastic Net alpha=0.05 · Synthetic data, high redundancy | OOB misclassification error: 14% -> 14% | No clear effect |
| Use Elastic Net (L1+L2) instead of LASSO under high redundancy | Regularized logistic regression · Synthetic data, high redundancy | stability: stability greater than 0.90 for lambda > 0.56 | Improved |
| Apply Stability Selection on top of LASSO | LASSO · Spambase, Boston housing, Sonar, Madelon | stability | Improved |
| Stability Selection vs LASSO, significance test at 5% | Stability Selection · Sonar and Madelon | stability significance | Mixed |
| Increase number of samples M | Stability estimator · Toy Bernoulli data d=100 | estimation error |Phi-hat - Phi|: within 5% for M <= 10, within 1% for M <= 100 | Improved |
Closest papers by meaning
- Estimating Conditional Mutual Information for Dynamic Feature Selection (ICLR 2024 · similarity 0.87)
- Building a stable classifier with the inflated argmax (NeurIPS 2024 · similarity 0.86)
- SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.86)
- Most Influential Subset Selection: Challenges, Promises, and Beyond (NeurIPS 2024 · similarity 0.86)
- Distributionally Robust Feature Selection (NeurIPS 2025 · similarity 0.85)
- Statistical Test for Feature Selection Pipelines by Selective Inference (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
