Research map / Label-efficient and robust learning / Recognition and long-tail

Hypernetworks for Dynamic Feature Selection

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Data augmentation and generalization bias · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hyper-DFS, a hypernetwork that generates subset-specific classifiers for dynamic feature selection, conditioned on a Set Transformer encoding of the observed-feature mask, and reports competitive or best AUAC-F1 across synthetic, tabular, and image benchmarks.

Problem

Dynamic feature selection must train predictors that work well across exponentially many observed-feature subsets; shared predictors compromise across subsets, while per-subset predictors are computationally prohibitive.

Approach

Replace a shared mask-conditioned predictor with a hypernetwork that generates subset-specific primary-network weights from a Set Transformer encoding of the knowledge status (with absence/presence embeddings), with a compressor, per-batch mask budgeting, and LR warm-up to stabilise training.

Key results

• HyperEns achieves the best average rank on synthetic (1.67) and tabular (4.33) benchmarks; single-model Hyper-DFS reaches average AUAC 71.11 on tabular vs 71.61 for HyperEns and 70.03 for VIP.
• On image patch benchmarks Hyper-DFS achieves the highest AUAC-F1 on four of five datasets, e.g. 81.19 on FashionMNIST and 73.39 on PCam.
• On held-out feature subsets, Hyper-DFS average AUAC is 64.06 on tabular (vs 46.55 for VIP) and 87.57 on image (vs 85.02 for VIP).

Surprising findings

• Hyperensembling over perturbed subset encodings yielded no measurable gain, indicating smoothness limits diversity.
• Multi-model baselines like Card. and Ensemble often did not consistently improve over strong single-model baselines on image datasets.

Limitations

• Training a hypernetwork is more expensive; full model has 0.68M-6.70M parameters versus 4.9k-11.3k for the generated primary predictor.
• Smoothness of the conditioning space limits diversity; hyperensembling gave no gain.
• Experiments did not have enough datasets to find statistically significant differences between methods.

Experimental setup

Datasets: Cube Diabetes Heart Disease Cirrhosis Wine Yeast Bank Marketing California housing MiniBooNE METABRIC MNIST Fashion MNIST

Benchmarks: Cube Synergistic Pairs Proxy Substitution Diabetes Heart Disease Cirrhosis Wine Yeast Bank Marketing California housing MiniBooNE METABRIC

Metrics: AUAC-F1 F1-macro

Baselines: CWCF DIME EDDI INVASE RePa SEFA VIP AACO CardinalDFS Ensemble MoE

Builds on: hypernetworks INVASE DIME VIP RePa AACO Set Transformer

Closest papers by meaning

• SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.87)
• Estimating Conditional Mutual Information for Dynamic Feature Selection (ICLR 2024 · similarity 0.85)
• When Dynamic Data Selection Meets Data Augmentation: Achieving Enhanced Training Acceleration (ICML 2025 · similarity 0.85)
• SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.85)
• Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection (ICML 2024 · similarity 0.84)
• Contextual Feature Selection with Conditional Stochastic Gates (ICML 2024 · similarity 0.84)