Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
PAGER: Accurate Failure Characterization in Deep Regression Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PAGER, a failure-characterization framework for deep regressors that combines anchored-network epistemic uncertainty with reverse-anchoring manifold non-conformity scores to sort test samples into ID, low, moderate and high risk regimes.
Problem
Detecting and characterizing failures in deep regression models, where epistemic uncertainty alone fails to track true risk under out-of-support and out-of-distribution shifts.
Approach
Uses anchored training for both forward-anchored uncertainty and a reverse-anchoring manifold non-conformity score (Score1, and an optimized input-projection variant Score2 with an anchored autoencoder regularizer), combined to bin test samples into risk regimes without calibration data.
Key results
- On 1D benchmarks PAGER (Score1) reduces FN on f1 to 5.61 vs 6.19 for the next best and on f2 to 0.0 vs 5.37 for Anchor UQ-only.
- On HD regression (Tails), PAGER achieves >50% reduction in FN and FP over baselines in many datasets, e.g., Camel FN 0.0 vs 10.53 for DEUP.
- On CIFAR-10 rotation, PAGER (FP 7.86, FN 3.34) is lower than DEUP (FP 15.22, FN 14.90) without calibration data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine uncertainty and manifold non-conformity scores rather than use uncertainty alone (ablation) | PAGER · CIFAR-10 rotation angle | FP: 12.54 -> 5.05 | Helped |
| combine uncertainty and manifold non-conformity scores rather than use non-conformity alone (ablation) | PAGER · CIFAR-10 rotation angle | FP: 13.08 -> 5.05 | Helped |
| use Score2 instead of Score1 (ablation) | PAGER · regression benchmarks (Gaps) | confusion between low-risk regimes (Clow) | Mixed |
| add anchored training (ablation) | regression models · CIFAR-10 rotation angle, chair yaw angle, and cell count | R2 (unobserved): 0.77 -> 0.81; 0.73 -> 0.75; 0.69 -> 0.72 | Helped |
| add post-hoc calibration data (ablation) | PAGER · DTI | FN: 6.67 -> 3.88 | Helped |
| add post-hoc calibration data (ablation) | PAGER · CIFAR-10 | FN: 3.34 -> 2.59 | Helped |
| train an anchored regression head with a frozen feature extractor (ablation) | PAGER · CIFAR-10 rotation angle prediction (Gaps) | failure-characterization metrics | Helped |
| use Score2 rather than Score1 under defocus blur and frost corruptions (ablation) | PAGER · CIFAR-10 rotation angle prediction | FP and FN | Helped |
Surprising findings
- Epistemic uncertainty alone is insufficient: low-uncertainty regions can carry high risk, and out-of-support samples may have low risk if the model extrapolates well.
- Anchor-UQ alone is a stronger baseline than DEUP in many 1D cases.
Limitations
- Risk-regime thresholds (25/75 quantiles) are demonstrative and chosen heuristically.
- Score2 is substantially more computationally expensive than Score1.
- Score1 alone does not fully reduce FN; MNC-only is unnormalized and insufficient by itself.
Experimental setup
Datasets: 1D benchmark functions Camel Levy Airfoil NO2 Kinematics Puma Boston Housing Ailerons Drug-Target Interactions Skillcraft chair yaw angle
Benchmarks: 1D benchmark functions Camel Levy Airfoil NO2 Kinematics Puma Boston Housing Ailerons DTI Skillcraft CIFAR-10 rotation
Metrics: FN FP C_low C_high R2
Baselines: DEUP DataSUITE Anchor UQ-only MNC-only
Builds on: anchored training (Thiagarajan et al., 2022) DEUP conformal prediction DataSUITE
Closest papers by meaning
- Beyond the Norms: Detecting Prediction Errors in Regression Models (ICML 2024 · similarity 0.86)
- Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models (ICML 2024 · similarity 0.84)
- Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers (NeurIPS 2024 · similarity 0.83)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.83)
- Fully Heteroscedastic Count Regression with Deep Double Poisson Networks (ICML 2025 · similarity 0.82)
- Typicalness-Aware Learning for Failure Detection (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.