Research map / Label-efficient and robust learning / Recognition and long-tail
A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition
Patricia Wollstadt, Sebastian Schmitt, Michael Wibral
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman I | true positive rate (%): 81.0 -> 100.0 | Improved |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman I | false negative rate (%): 18.4 -> 0.0 | Improved |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman II | true positive rate (%): 56.8 -> 50.5 | Hurt |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman II | false positive rate (%): 0.8 -> 0.2 | Improved |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman III | true positive rate (%): 3.3 -> 3.5 | No clear effect |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Friedman (all three models) | true positive rate (%) | Mixed |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Runge et al. 2015 model | true positive rate (%): 54.7 -> 99.6 | Improved |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Runge et al. 2015 model | false positive rate (%): 7.5 -> 3.0 | Improved |
| use CMI instead of MI as inclusion criterion (ablation) | sequential forward selection · Runge et al. 2015 model | false negative rate (%): 45.3 -> 0.4 | Improved |
| choose CMI forward selection over LARS/RF forward selection with fixed feature count | feature selection · Friedman II and III | false positives from nuisance variables | Improved |
| select feature set by CMI criterion instead of using all features (ablation) | KNN predictor (k=5) · Friedman II | MAE | Improved |
| increase signal-to-noise by lowering noise sigma | PID/CMI estimates · Experiment II statistical models | absolute CMI, MI and PID values | Improved |
Closest papers by meaning
- Estimating Conditional Mutual Information for Dynamic Feature Selection (ICLR 2024 · similarity 0.89)
- Variational Information Maximization for Feature Selection (NeurIPS 2016 · similarity 0.88)
- SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.86)
- Contextual Feature Selection with Conditional Stochastic Gates (ICML 2024 · similarity 0.85)
- A Differentiable Rank-Based Objective for Better Feature Learning (ICLR 2025 · similarity 0.84)
- Partial Information Decomposition via Normalizing Flows in Latent Gaussian Distributions (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
