Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
A Unified View of FANOVA: A Comprehensive Bayesian Framework for Component Selection and Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hierarchical Bayesian global-local GP framework for FANOVA models with multi-level sparsity (Dirichlet, exponential, horseshoe mixing), unifying prior Bayesian and COSSO-like deterministic methods and showing a Bayesian counterpart of COSSO.
Problem
Component selection and estimation in functional ANOVA regression models, where the number of functional components grows as O(D^r) and only a few are relevant; existing deterministic COSSO-type methods lack uncertainty quantification and Bayesian methods are disconnected from them.
Approach
A hierarchical GP model with global-local shrinkage per interaction order, including a non-separable Dirichlet mixing prior on local scales that gives a continuous approximation of discrete spike-and-slab-like sparsity, plus covariate selection layers; shows the MAP under exponential mixing with equal global scales recovers the COSSO solution.
Key results
- On simulated data the Dirichlet mixing gives superior selection, with fewer false positives than SS-ANOVA and COSSO while keeping comparable RMSE.
- On the Housing UCI dataset, sparse models reduce the number of active components by half versus SS-ANOVA while achieving comparable RMSE.
- Covariate selection with soft shrinkage in the r=2, D=30 simulation reduces false positives from 60.3 (no covariate selection) to 18.8 while RMSE improves from 1.66 to 1.46.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add covariate selection with hard shrinkage (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | False Positives: 60.30±10.23 -> 8.30±8.87 | Helped |
| Add covariate selection with hard shrinkage (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | False Negatives: 1.40±0.30 -> 2.00±0.50 | Hurt |
| Add covariate selection with hard shrinkage (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | RMSE: 1.66 ± 0.12 -> 1.43 ± 0.42 | Helped |
| Add covariate selection with soft shrinkage (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | False Positives: 60.30±10.23 -> 18.81±11.35 | Helped |
| Add covariate selection with soft shrinkage (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | True Positives: 5.60±0.30 -> 5.40±0.40 | No clear effect |
| Use hard rather than soft shrinkage for covariate selection (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | False Negatives: 1.60±0.40 -> 2.00±0.50 | Hurt |
| Use soft rather than hard shrinkage for covariate selection (ablation) | Proposed FANOVA model · Simulated data (r = 2, D = 30) | False Negatives: 2.00±0.50 -> 1.60±0.40 | Helped |
| Use sampling rather than plug-in estimation (ablation) | Proposed Bayesian FANOVA models · Simulated data | F1 score | Hurt |
| Use sampling rather than plug-in estimation (ablation) | Proposed Bayesian FANOVA models · Simulated data | Coverage probability | Helped |
| Use a single global scale across interaction orders instead of separate order-specific global scales (ablation) | Bayesian FANOVA model · Demand | Number of relevant components | Hurt |
| Use a single local scale rather than heterogeneous local scales (ablation) | Proposed FANOVA model · UCI datasets | Number of active components | Hurt |
| Tune mixing-model weights using the SS-ANOVA estimator (ablation) | Adaptive mixing models · UCI datasets | RMSE | Mixed |
Surprising findings
- Using a single global scale across interaction orders can introduce spurious interaction components; per-order global scales avoid this in the Demand dataset.
- Plug-in MAP estimation gave sparser solutions and higher F1 than fully Bayesian sampling, while sampling gave more accurate uncertainty intervals.
Limitations
- Orthogonality of functional subspaces implicitly assumes independent covariates; non-independent covariates can lead to non-identifiability.
- Dirichlet model converges more slowly than independent mixing models, and may not optimize accuracy when no strong sparsity prior exists.
- Hard covariate-selection shrinkage can miss components whose relevance depends on an interaction with a non-relevant main effect.
Experimental setup
Datasets: UCI regression datasets (including Housing, Demand) simulated data (independent uniform, compound symmetry, trimmed autoregressive)
Benchmarks: UCI regression datasets (5-fold CV) simulated FANOVA data
Metrics: RMSE F1 score number of active components (NAC) 95% credible interval coverage
Baselines: SS-ANOVA COSSO ACOSSO MARS RBF-Ok (additive models with orthogonal kernel)
Builds on: COSSO SS-ANOVA Bayesian global-local shrinkage (Tang et al., 2018) adaptive LASSO
Closest papers by meaning
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.85)
- Simultaneous identification of models and parameters of scientific simulators (ICML 2024 · similarity 0.85)
- Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm (NeurIPS 2025 · similarity 0.84)
- Bivariate Causal Discovery using Bayesian Model Selection (ICML 2024 · similarity 0.84)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.83)
- Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.