Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Integral Imprecise Probability Metrics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Integral Imprecise Probability Metrics (IIPMs), a Choquet-integral generalisation of IPMs to capacities, and derives a new epistemic uncertainty measure (MMI) from comparing a lower probability with its conjugate, validated on selective classification.
Problem
Comparing imprecise probability models (capacities, lower probabilities) and quantifying epistemic uncertainty within a single imprecise probability model lack a principled integral-based metric framework.
Approach
Replace Lebesgue integrals in IPMs with Choquet integrals to define IIPMs on capacities; comparing a lower probability with its conjugate upper probability yields the maximum mass interval (MMI) epistemic uncertainty measure, which satisfies key credal UQ axioms and has an O(K) upper bound (MMI-Lin).
Key results
- MMI and MMI-Lin consistently outperform E-Diff on accuracy-rejection curves across 7, 10 and 100-class tasks.
- MMI performs almost identically to the generalised Hartley measure (GH) across tasks.
- For large K, MMI-Lin (O(K)) remains tractable where exact MMI and GH (O(2^K)) are intractable, and still outperforms E-Diff.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MMI instead of entropy difference (E-Diff) for selective classification | Selective classification UQ measures · Classification tasks with 7, 10, and 100 classes | AR curve AUC | Helped |
| Use MMI instead of Generalised Hartley (GH) for selective classification | Selective classification UQ measures · Classification tasks with 7, 10, and 100 classes | AR curve AUC | No clear effect |
| Use the linear-time MMI upper bound (MMI-Lin) instead of E-Diff | Selective classification UQ measures · Large-class classification tasks | AR curve AUC | Helped |
Surprising findings
- Empirically, MMI and GH perform near-identically, suggesting their equivalence may hold for arbitrary K.
Limitations
- No sampling-based estimator for IIPMs, unlike classical IPMs.
- Nonparametric estimation of IIPMs is not yet developed.
- Lower probabilities used in experiments are constructed from an ensemble of 10 predictors, with centroid prediction.
Experimental setup
Datasets: UCI datasets CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100 UCI datasets
Metrics: accuracy-rejection curve area under the curve (AUC)
Baselines: E-Diff GH MMI-Lin
Builds on: integral probability metric Choquet integral generalised Hartley measure entropy difference
Closest papers by meaning
- Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.87)
- Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods? (ICML 2024 · similarity 0.86)
- Second-Order Uncertainty Quantification: A Distance-Based Approach (ICML 2024 · similarity 0.86)
- Hierarchical Integral Probability Metrics: A distance on random probability measures with low sample complexity (ICML 2024 · similarity 0.85)
- Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference (NeurIPS 2026 · similarity 0.84)
- Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.