Research map / Probabilistic and scientific modeling / Variational and generative sampling
Projection Pursuit Density Ratio Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ppDRE, a density ratio estimator that approximates the ratio as a product of projection pursuit univariate functions fit iteratively with linear sieves, mitigating the curse of dimensionality, with consistency and rate results.
Problem
Density ratio estimation suffers from model misspecification in parametric methods and the curse of dimensionality in nonparametric linear sieve and kernel methods when data dimension is large.
Approach
Approximate the density ratio by a multiplicative product of projection pursuit functions f_k(a_k^T x), each estimated iteratively via a univariate linear sieve with an L2-type loss and ridge penalty, with directions optimized by gradient descent (Adam) and establishing consistency and convergence rates.
Key results
- In stabilized weights estimation, ppDRE achieved the lowest median RMSE across dimensions 2 to 100 (e.g., 3.835 vs 4.216 for uLSIF at d=10).
- On covariate shift benchmarks, ppDRE obtained the lowest NMSE on Abalone (0.514), Billboard Spotify (0.559), Computer Activity (0.199) and Diamond Prices (0.133).
- For mutual information estimation at p=20 and rho=0.8, ppDRE achieved MAE 2.966 versus 7.185 for the best alternative (classification).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous ADRF | ASE: 0.106(0.026) -> 0.091(0.025) | Helped |
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous QDRF, τ=0.1 | ASE: 0.307(0.053) -> 0.338(0.066) | Hurt |
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous QDRF, τ=0.25 | ASE: 0.125(0.026) -> 0.124(0.029) | Helped |
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous QDRF, τ=0.5 | ASE: 0.043(0.015) -> 0.041(0.016) | Helped |
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous QDRF, τ=0.75 | ASE: 0.088(0.025) -> 0.079(0.022) | Helped |
| Use ppDRE instead of the strongest listed baseline | Dose-response estimation on IHDP-continuous · IHDP-continuous QDRF, τ=0.9 | ASE: 0.267(0.061) -> 0.266(0.060) | Helped |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=2, ρ=0.2 | MAE: 0.043(0.000) -> 0.051(0.044) | Hurt |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=2, ρ=0.8 | MAE: 0.169(0.019) -> 0.069(0.015) | Helped |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=10, ρ=0.2 | MAE: 0.199(0.007) -> 0.164(0.038) | Helped |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=10, ρ=0.8 | MAE: 3.838(1.912) -> 0.272(0.183) | Helped |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=20, ρ=0.2 | MAE: 0.164(0.132) -> 0.074(0.098) | Helped |
| Use ppDRE instead of the strongest listed baseline | Mutual information estimation · Mutual information, p=20, ρ=0.8 | MAE: 8.629(2.577) -> 2.966(1.089) | Helped |
Surprising findings
- Conventional linear sieve methods (uLSIF, KLIEP) degrade sharply as dimension grows from 2 to 10, and nnDRE showed very high variance in some MI and covariate shift settings.
Limitations
- Computation time grows with dimension and number of projections K, which is tuned by cross-validation.
- Theory relies on compactness, smoothness and sieve regularity assumptions; extension to independence testing left for future work.
Experimental setup
Datasets: IHDP Abalone Billboard Spotify Cancer Mortality Computer Activity Diamond Prices
Benchmarks: IHDP-continuous Abalone Billboard Spotify Cancer Mortality Computer Activity Diamond Prices
Metrics: RMSLE RMSE ASE MAE NMSE
Baselines: uLSIF KLIEP probabilistic classification (LightGBM) D3-LHSS fDRE RRND nnDRE
Builds on: KLIEP uLSIF projection pursuit
Closest papers by meaning
- Density Ratio Estimation with Conditional Probability Paths (ICML 2025 · similarity 0.86)
- Bounds on $L_p$ Errors in Density Ratio Estimation via $f$-Divergence Loss Functions (ICLR 2025 · similarity 0.85)
- Classification Diffusion Models: Revitalizing Density Ratio Estimation (NeurIPS 2024 · similarity 0.84)
- Overcoming Saturation in Density Ratio Estimation by Iterated Regularization (ICML 2024 · similarity 0.84)
- Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning (ICML 2025 · similarity 0.83)
- Density Ratio Estimation with Doubly Strong Robustness (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.