Research map / Label-efficient and robust learning / Recognition and long-tail
Risk Controlled Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Risk Controlled Image Retrieval (RCIR), which adapts retrieval set size using heuristic uncertainty and a Hoeffding-based calibration scale so retrieval sets contain a true nearest neighbor with a user-specified probability.
Problem
Image retrieval systems prioritize accuracy and offer only heuristic uncertainty with fixed-size retrieval sets, providing no statistical guarantee that retrieved sets contain true nearest neighbors at a required reliability level.
Approach
Scale retrieval set size per query as a function of normalized heuristic uncertainty, with a single scale kappa calibrated offline using a Hoeffding upper confidence bound on the miss-all-neighbors risk, giving a provable coverage guarantee agnostic to data distribution and model.
Key results
- RCIR variants keep empirical risk below the predefined level alpha across datasets, whereas heuristic-uncertainty-only baselines (MCD, BTL, Deep Ensemble) exceed alpha for alpha < 0.4.
- Deep Ensemble achieves the best Recall@1 among the compared retrieval systems on the evaluated datasets.
- RCIR retrieval set sizes are similar to fixed-size counterparts at the same recall@K, and a smaller error rate delta yields larger, more conservative retrieval sets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RCIR risk control to uncertainty-aware retrieval | BTL, MCD, and Deep Ensemble · CAR-196, CUB-200, Pittsburgh, and ChestX-Det test sets | empirical risk | Helped |
| Replace MCD's uncertainty estimator with a random uncertainty estimator (ablation) | MCD-based RCIR · Risk-control experiment | empirical risk | No clear effect |
| Replace MCD's uncertainty estimator with a random uncertainty estimator (ablation) | MCD-based RCIR · Risk-control experiment | average retrieval set size | Hurt |
| Use a smaller error rate δ | RCIR · Pittsburgh | average retrieval set size | Hurt |
| Use a smaller risk level α | RCIR · Pittsburgh | retrieval set size | Hurt |
| Use BTL instead of Deterministic | Image retrieval feature extractor · Most datasets | Recall@1 | Helped |
| Use MCD instead of Deterministic | Image retrieval feature extractor · Various test sets | Recall@1 | Mixed |
| Use Deep Ensemble instead of other image retrieval systems | Image retrieval · Various test sets | Recall@1 | Helped |
| Use heuristic uncertainty alone instead of RCIR risk control | MCD, BTL, and Deep Ensemble · Test sets with α<0.4 | empirical risk | Hurt |
Surprising findings
- A randomly assigned uncertainty still yields valid risk control under RCIR, but produces larger retrieval sets because kappa is computed suboptimally.
Limitations
- A poorly calibrated uncertainty estimator increases average retrieval set size.
- Heuristic uncertainty is not well calibrated on the tested datasets, and the method does not reduce risk beyond the controlled level.
- Guarantee relies on exchangeability between calibration and test data.
Experimental setup
Datasets: CUB-200 Stanford CAR-196 Pittsburgh ChestX-Det
Benchmarks: CUB-200 Stanford CAR-196 Pittsburgh ChestX-Det
Metrics: recall@1 reliability diagram empirical risk retrieval set size
Baselines: BTL MCD Deep Ensemble Deterministic retrieval Random uncertainty
Builds on: Monte Carlo Dropout Deep Ensembles Bayesian triplet loss (BTL) conformal prediction distribution-free risk controlling prediction sets
Closest papers by meaning
- Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization (ICLR 2024 · similarity 0.85)
- From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation (ICLR 2025 · similarity 0.82)
- Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.82)
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.82)
- REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability (AAAI 2025 · similarity 0.81)
- Data Roaming and Quality Assessment for Composed Image Retrieval (AAAI 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.