Research map / Label-efficient and robust learning / Test-time adaptation
Learning for Transductive Threshold Calibration in Open-World Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines open-world threshold calibration and proposes OpenGCN, a GAT-based transductive method that predicts pairwise connectivity and densities on unlabeled test graphs to pick distance thresholds for target TPR/TNR.
Problem
Calibrating distance thresholds in deep metric learning to reach target TPR or TNR on unseen open-world test classes whose distance distributions differ from the calibration data; existing posthoc calibration is inductive and fails under distribution shift.
Approach
Treat threshold calibration as transductive inference: a GAT on fully connected graphs of unlabeled test embeddings predicts pairwise connectivity plus two density metrics (average and neighborhood), from which TPR/TNR curves are estimated; training is pretrain on closed-set data then fine-tune only the MLP head on a disjoint open-world calibration set.
Key results
- SameDist: OpenGCN reduces global MAE_comb by 59.30% (Cars), 66.49% (CUB), 59.15% (iNat) relative to the best baseline.
- ShiftDist on Cars with 13 corruptions: average 55.03% error reduction over the best baseline.
- DiffDist cross-dataset (iNat-to-CUB): error reduced by 87.65%; average 43.99% reduction across DiffDist settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OpenGCN instead of Hi-LANDER | ResNet-50 distance calibration · Cars-196 SameDist | MAEcomb: 1.29e-2 -> 5.25e-3 | Helped |
| use OpenGCN instead of Hi-LANDER | ResNet-50 distance calibration · CUB SameDist | MAEcomb: 1.94e-2 -> 6.50e-3 | Helped |
| use OpenGCN instead of isotonic regression | ResNet-50 distance calibration · iNaturalist-2018 SameDist | MAEcomb: 1.18e-2 -> 4.82e-3 | Helped |
| use OpenGCN instead of DBSCAN | ResNet-50 distance calibration · Cars-196 sketch DiffDist | MAEcomb: 5.16e-2 -> 3.54e-2 | Helped |
| use OpenGCN instead of Platt scaling | ResNet-50 distance calibration · CUB cross-dataset DiffDist | MAEcomb: 1.15e-1 -> 1.42e-2 | Helped |
| use OpenGCN instead of Platt scaling | ResNet-50 distance calibration · iNaturalist-2018 long-tail DiffDist | MAEcomb: 2.09e-2 -> 1.82e-2 | Helped |
| add average-density prediction to connectivity prediction (ablation) | OpenGCN · iNaturalist-2018 SameDist | MAEcomb: 6.25e-3 -> 5.37e-3 | Helped |
| add neighborhood-density prediction to connectivity prediction (ablation) | OpenGCN · iNaturalist-2018 SameDist | MAEcomb: 6.25e-3 -> 5.12e-3 | Helped |
| add both average- and neighborhood-density predictions (ablation) | OpenGCN · iNaturalist-2018 SameDist | MAEcomb: 6.25e-3 -> 4.82e-3 | Helped |
| fine-tune the MLP on open-world calibration data after pretraining (ablation) | OpenGCN · Cars-196 SameDist | MAEcomb: 2.90e-2 -> 5.25e-3 | Helped |
| fine-tune the MLP on open-world calibration data after pretraining (ablation) | OpenGCN · CUB SameDist | MAEcomb: 2.52e-2 -> 6.50e-3 | Helped |
| fine-tune the MLP on open-world calibration data after pretraining (ablation) | OpenGCN · iNaturalist-2018 SameDist | MAEcomb: 3.55e-2 -> 4.82e-3 | Helped |
Surprising findings
- Clustering-based DBSCAN performs worse than traditional posthoc calibration methods on most settings.
- Hi-LANDER beats posthoc baselines on Cars and CUB but underperforms them on iNaturalist.
Limitations
- Computationally less efficient and more prone to over-parameterization than posthoc calibration methods.
- Not calibration-data-free; still requires a labeled calibration set plus closed-world training data.
Experimental setup
Datasets: iNaturalist-2018 CUB-200 Cars-196
Benchmarks: iNaturalist-2018 CUB-200 Cars-196
Metrics: MAE_comb (global error) absolute error at target TPR/TNR (point-wise)
Baselines: Platt scaling Beta calibration Isotonic regression Histogram calibration DBSCAN Hi-LANDER
Builds on: Hi-LANDER Platt scaling temperature scaling Graph Attention Network
Closest papers by meaning
- Improving GNN Calibration with Discriminative Ability: Insights and Strategies (AAAI 2024 · similarity 0.85)
- Threshold-Consistent Margin Loss for Open-World Deep Metric Learning (ICLR 2024 · similarity 0.84)
- Open-Vocabulary Calibration for Fine-tuned CLIP (ICML 2024 · similarity 0.84)
- Understanding and Mitigating Under-Confidence in GNNs from the Final Layer (NeurIPS 2026 · similarity 0.83)
- HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration (NeurIPS 2026 · similarity 0.83)
- MaxEnt Loss: Calibrating Graph Neural Networks under Out-of-Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.