Research map / Label-efficient and robust learning / Recognition and long-tail
Stochastic Encodings for Active Feature Acquisition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEFA, a supervised stochastic-latent-variable encoder-predictor that scores features by probability-weighted expected latent gradient importance for active feature acquisition, outperforming RL and CMI-based baselines.
Problem
Active feature acquisition: choosing, per test instance and sequentially, which features to measure. Existing RL methods suffer training difficulties, while greedy conditional mutual information (CMI) maximization is myopic and favors reducing low-probability classes.
Approach
Train a stochastic latent encoder-predictor with a supervised NLL plus information-bottleneck KL loss, then score each unobserved feature by the expected, normalized norm of the predictor's latent-gradient components for that feature, weighted by current class probabilities, averaged over latent samples that represent possible unobserved realizations.
Key results
- On synthetic Syn 1-3, SEFA needs 4.017, 4.099, and 5.084 acquisitions versus 4.079, 4.484 (GDFS), and 5.667 for the best baselines (DIME/GDFS), close to the optimal 3/3/5.
- SEFA attains the highest average metric on all real datasets, e.g. 0.761 on MNIST vs 0.740 for Opportunistic RL and 0.843 on TCGA vs 0.838 for Opportunistic RL.
- Ablations show computing the objective in feature space is the most damaging change (e.g. MNIST 0.697 vs 0.761), followed by deterministic encoders and single acquisition samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SEFA instead of the strongest reported baseline | active feature acquisition · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.079 ± 0.064 -> 4.017 ± 0.003; 4.581 ± 0.217 -> 4.099 ± 0.009; 5.587 ± 0.201 -> 5.084 ± 0.026 | Helped |
| use SEFA instead of the strongest reported baseline | active feature acquisition · Cube, Bank Marketing, California Housing, MiniBooNE, MNIST, Fashion MNIST, METABRIC, and TCGA | average acquisition evaluation metric: 0.901 ± 0.001 -> 0.904 ± 0.001; 0.910 ± 0.000 -> 0.919 ± 0.001; 0.661 ± 0.002 -> 0.676 ± 0.005; 0.954 ± 0.000 -> 0.957 ± | Helped |
| add latent-space regularization (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.520 ± 0.082 -> 4.017 ± 0.003; 4.578 ± 0.109 -> 4.099 ± 0.009; 5.716 ± 0.104 -> 5.084 ± 0.026 | Helped |
| use multiple acquisition samples (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.683 ± 0.030 -> 4.017 ± 0.003; 4.862 ± 0.035 -> 4.099 ± 0.009; 5.700 ± 0.027 -> 5.084 ± 0.026 | Helped |
| use multiple training samples (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.421 ± 0.174 -> 4.017 ± 0.003; 4.713 ± 0.160 -> 4.099 ± 0.009; 5.188 ± 0.107 -> 5.084 ± 0.026 | Helped |
| use stochastic rather than deterministic encoders (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.593 ± 0.236 -> 4.017 ± 0.003; 4.773 ± 0.228 -> 4.099 ± 0.009; 5.744 ± 0.034 -> 5.084 ± 0.026 | Helped |
| calculate the acquisition objective in latent rather than feature space (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 5.111 ± 0.070 -> 4.017 ± 0.003; 5.461 ± 0.128 -> 4.099 ± 0.009; 5.977 ± 0.052 -> 5.084 ± 0.026 | Helped |
| normalize acquisition scores (ablation) | SEFA · Syn 1, Syn 2, and Syn 3 | number of acquisitions: 4.036 ± 0.009 -> 4.017 ± 0.003; 4.104 ± 0.008 -> 4.099 ± 0.009; 5.101 ± 0.015 -> 5.084 ± 0.026 | Helped |
| remove probability weighting (ablation) | SEFA · MNIST, Fashion MNIST, METABRIC, and TCGA | average acquisition accuracy: 0.751 ± 0.001 vs 0.761 ± 0.001; 0.694 ± 0.001 vs 0.721 ± 0.000; 0.698 ± 0.003 vs 0.709 ± 0.003; 0.832 ± 0.001 vs 0.843 ± | Hurt |
| remove probability weighting (ablation) | SEFA · Bank Marketing and MiniBooNE | average AUROC: 0.919 ± 0.001 vs 0.919 ± 0.001; 0.957 ± 0.000 vs 0.957 ± 0.000 | No clear effect |
| remove acquisition-score normalization (ablation) | SEFA · Cube, MNIST, and TCGA | average acquisition accuracy: 0.904 ± 0.001 -> 0.905 ± 0.000; 0.761 ± 0.001 -> 0.762 ± 0.001; 0.843 ± 0.002 -> 0.844 ± 0.002 | Mixed |
| remove acquisition-score normalization (ablation) | SEFA · Fashion MNIST | average acquisition accuracy: 0.721 ± 0.000 -> 0.718 ± 0.001 | Hurt |
Surprising findings
- Generative-model CMI estimators (ACFlow, EDDI, VAE) perform worse than a fixed global feature ordering on several datasets, showing inaccurate CMI estimation hurts greedy acquisition.
- On MiniBooNE, the fixed ordering MLP is second best, ahead of the active methods that actively acquire features.
Limitations
- Applies only to classification, not regression, because the acquisition objective needs class probabilities.
- Requires multiple latent samples at inference, increasing memory requirements compared with RL baselines.
Experimental setup
Datasets: Syn 1-3 (synthetic) Cube Bank Marketing California Housing MiniBooNE MNIST Fashion MNIST METABRIC TCGA
Benchmarks: Syn 1 Syn 2 Syn 3 Cube Bank Marketing California Housing MiniBooNE MNIST Fashion MNIST METABRIC TCGA
Metrics: number of acquisitions to correct features average AUROC/accuracy during acquisition accuracy
Baselines: ACFlow DIME EDDI Fixed MLP GDFS GSMRL Opportunistic RL Random VAE
Builds on: Opportunistic learning DIME GDFS EDDI ACFlow
Closest papers by meaning
- Information Templates: A New Paradigm for Intelligent Active Feature Acquisition (NeurIPS 2026 · similarity 0.89)
- Active Learning with LLMs for Partially Observed and Cost-Aware Scenarios (NeurIPS 2024 · similarity 0.87)
- Active feature acquisition via explainability-driven ranking (ICML 2025 · similarity 0.87)
- Estimating Conditional Mutual Information for Dynamic Feature Selection (ICLR 2024 · similarity 0.84)
- Generator Assisted Mixture of Experts for Feature Acquisition in Batch (AAAI 2024 · similarity 0.83)
- Contextual Active Model Selection (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.