Research map / Probabilistic and scientific modeling / Variational and generative sampling
Density-Ratio Losses for Post-Hoc Learning to Defer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes post-hoc learning-to-defer via density-ratios between model and expert ideal distributions (KL-based), estimated with a DRE-to-CPE reduction yielding DR CPE losses whose deferral rate is tuned by thresholding a scorer.
Problem
Post-hoc learning to defer requires a deferral rule that can be adjusted to different deferral rates without retraining, and existing expert-comparison or score-based methods are fragile under specialized experts and distribution shift.
Approach
Define deferral as thresholding the density ratio between model and expert KL-ideal distributions (divergence-regularized reweightings of the data), and learn it via DRE-to-CPE reduction so that the joint ideal DR CPE loss is an unbiased expectation under P; thresholding the learned scorer gives tunable deferral rates.
Key results
- DR CPE is competitive with baselines on clean data and performs best to second best across corrupted settings.
- Estimate-Diff01 beats DR CPE on CIFAR-100 long-tail and DermaMNIST specialist settings but fails on PathMNIST specialist.
- Conf and TwoStage-Exp perform extremely poorly on the DermaMNIST specialist setting.
Surprising findings
- The expert-comparison estimator Estimate-Diff01 critically fails in the PathMNIST specialist setting, which the authors attribute to overfitting.
- The marginal DR CPE loss is not practically estimable, so the joint variant is used, with a provable gap to the marginal loss that depends on label variance.
Limitations
- No consistency guarantees, unlike surrogate-loss L2D approaches.
- Temperature gamma has a non-trivial effect on performance.
- Does not handle fairness or multi-expert settings.
Experimental setup
Datasets: CIFAR-100 MedMNIST (DermaMNIST) MedMNIST (PathMNIST)
Benchmarks: CIFAR-100 corrupted (label noise, long-tail, specialization) DermaMNIST corrupted variants PathMNIST corrupted variants
Metrics: accuracy-deferral trade-off
Baselines: Conf TwoStage-Exp Estimate-Diff01 Estimate-MaxProb
Builds on: Chow's rule learning to reject with ideal distributions density-ratio estimation via class-probability estimation generalized cross-entropy
Closest papers by meaning
- Exploiting Human-AI Dependence for Learning to Defer (ICML 2024 · similarity 0.86)
- Rejection via Learning Density Ratios (NeurIPS 2024 · similarity 0.85)
- Regression with Multi-Expert Deferral (ICML 2024 · similarity 0.84)
- A Two-Stage Learning-to-Defer Approach for Multi-Task Learning (ICML 2025 · similarity 0.84)
- Mastering Multiple-Expert Routing: Realizable $H$-Consistency and Strong Guarantees for Learning to Defer (ICML 2025 · similarity 0.83)
- A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.