Research map / Probabilistic and scientific modeling / Variational and generative sampling
Density Ratio Estimation with Doubly Strong Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes two outlier-robust density ratio estimation methods (Weighted DRE via unnormalized KL and γ-DRE via γ-divergence) with weight functions, claiming doubly strong robustness to contamination of both reference and target datasets.
Problem
Density ratio estimation is unreliable when outliers contaminate the reference and/or target distributions, and prior robust DRE (Trimmed DRE) only handles reference contamination and assumes outliers have large density ratio values.
Approach
Introduce a weight function into the Bregman/UKL and γ-divergence objectives for log-linear density ratio models so outlier regions are down-weighted; the γ-divergence version removes the normalizing constant from estimation, yielding a DC problem, and theory shows bias is O(ν) regardless of contamination ratios.
Key results
- In the double contamination setting, Table 2 shows Weighted DRE and γ-DRE MSE values (e.g., 0.03 and 0.013 at λp=λq=0) far below DRE (~5.4) and Trimmed DRE (~5.6).
- γ-DRE achieved smaller MSE than Weighted DRE in all tested settings of the synthetic experiment.
- On HASC change detection, Weighted DRE and γ-DRE avoided false detections at outliers that DRE and Trimmed DRE flagged, though they missed one change point around time 80.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Weighted DRE with a weight function to downweight outliers in both datasets | DRE · Double contamination, θ1,2=0.0 | MSE: 5.3757 -> 0.0299 | Helped |
| Use γ-DRE, which avoids estimating the normalizing term | DRE · Double contamination, θ1,2=0.0 | MSE: 5.3757 -> 0.0131 | Helped |
| Use γ-DRE instead of Weighted DRE | Robust DRE · Double contamination, θ1,2=1.6 | MSE: 0.9466 -> 0.3042 | Helped |
| Use Weighted DRE instead of DRE | DRE · Clean, θ1,2=1.6 | MSE: 1.5608 -> 0.7467 | Helped |
| Use γ-DRE instead of DRE | DRE · Clean, θ1,2=1.6 | MSE: 1.5608 -> 0.2931 | Helped |
Surprising findings
- Weighted DRE and γ-DRE missed a true change point (between 'stair down' and 'walk') that DRE and Trimmed DRE detected, because it lacked an apparent change in the raw data.
Limitations
- γ-DRE is a DC problem and requires more computation than the convex Weighted DRE.
- Theoretical robustness relies on Assumption 3.1/3.4 that the weight function suppresses outlier distributions (w must decay faster than h and the density ratio model).
- Experiments are small-scale (100 samples per dataset) with synthetic low-dimensional settings and one real change-detection dataset.
Experimental setup
Datasets: synthetic normal distributions with contamination Human Activity Sensing Consortium (HASC) Challenge 2011 accelerometer data
Benchmarks: synthetic precision matrix difference estimation (MSE) HASC 2011 change detection
Metrics: mean squared error qualitative change detection
Baselines: DRE Trimmed DRE
Builds on: Trimmed DRE gamma-divergence unnormalized Kullback-Leibler divergence KLIEP
Closest papers by meaning
- Trimmed Density Ratio Estimation (NeurIPS 2017 · similarity 0.88)
- Bounds on $L_p$ Errors in Density Ratio Estimation via $f$-Divergence Loss Functions (ICLR 2025 · similarity 0.87)
- Linking losses for density ratio and class-probability estimation (ICML 2016 · similarity 0.87)
- Outlier-Robust Distributionally Robust Optimization via Unbalanced Optimal Transport (NeurIPS 2024 · similarity 0.86)
- Sobolev Space Regularised Pre Density Models (ICML 2024 · similarity 0.84)
- Binary Losses for Density Ratio Estimation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.