Research map / Learning theory and algorithms / Prediction and decision losses
Generator Assisted Mixture of Experts for Feature Acquisition in Batch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a generator-assisted mixture-of-experts method for batch feature acquisition that uses random-hyperplane LSH buckets, a VAE feature generator, and a greedy set-function optimizer with approximation guarantees.
Problem
Feature acquisition when features must be queried in a single batch (no sequential feedback) under a per-instance oracle query budget, with heterogeneous observed feature subsets across instances.
Approach
Uses a feature generator to supply some queried features synthetically, partitions instances with random-hyperplane LSH into buckets, trains a mixture of experts per bucket, and optimizes a tractable upper-bound surrogate via greedy selection under new (m,M)-partial monotonicity and (gamma,Gamma)-weak submodularity properties.
Key results
- Outperforms six batch-adapted baselines (JAFA, EDDI, ACFlow, GSM, CwCF, DiFA) in accuracy vs. oracle query cost on all four datasets, reported as statistically significant (Welch's t-test, p < 0.01).
- Generator ablation: achieves 3-5x cost savings at the same accuracy compared to the variant that queries all features from the oracle (V = U).
- Random hyperplane clustering gives bucket-size ratio of 0.21 on DP vs. 0.014 (K-means) and 0.003 (GMM).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Generator Assisted Mixture of Experts instead of the batch baselines | Feature acquisition · DP, MNIST, CIFAR100, TinyImagenet | classification accuracy at a fixed oracle-query budget | Helped |
| Use the full generator-assisted method instead of the V=U variant (ablation) | Generator Assisted Mixture of Experts · Four datasets | oracle-query cost at the same accuracy: 3--5x | Helped |
| Use the full generator-assisted method instead of V=empty (ablation) | Generator Assisted Mixture of Experts · Four datasets | classification accuracy at a fixed oracle-query budget | Mixed |
| Use the full generator-assisted method instead of V=U (ablation) | Generator Assisted Mixture of Experts · Four datasets | classification accuracy at a fixed oracle-query budget | Mixed |
| Use random hyperplane clustering instead of K-means or Gaussian mixture clustering (ablation) | Generator Assisted Mixture of Experts · DP and CIFAR100 | classification accuracy across oracle-query budgets | Helped |
| Use random hyperplane clustering instead of K-means or GMM (ablation) | Generator Assisted Mixture of Experts · DP | bucket-size ratio (minimum/maximum): 0.21 vs. 0.014 and 0.003 | Helped |
Surprising findings
- Authors find K-means and GMM produce highly imbalanced buckets, with inter-cluster cosine similarities of at least 0.92, which they attribute to aggregate-similarity objectives.
Limitations
- null
Experimental setup
Datasets: DP (disease prediction) MNIST CIFAR100 TinyImagenet
Benchmarks: DP (disease prediction) MNIST CIFAR100 TinyImagenet
Metrics: classification accuracy average oracle queries per instance
Baselines: JAFA EDDI ACFlow GSM CwCF DiFA
Builds on: JAFA EDDI ACFlow beta-VAE
Closest papers by meaning
- Active feature acquisition via explainability-driven ranking (ICML 2025 · similarity 0.83)
- Stochastic Encodings for Active Feature Acquisition (ICML 2025 · similarity 0.83)
- Information Templates: A New Paradigm for Intelligent Active Feature Acquisition (NeurIPS 2026 · similarity 0.83)
- Divide and not forget: Ensemble of selectively trained experts in Continual Learning (ICLR 2024 · similarity 0.83)
- Diversified Batch Selection for Training Acceleration (ICML 2024 · similarity 0.83)
- Toward Efficient Inference for Mixture of Experts (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.