Research map / Vision-language and multimodal / Human and embodied modeling
Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Prediction
Aditya Sarkar, Yi Li, Jiacheng Cheng, Shlok Kumar Mishra, Nuno Vasconcelos
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add memory retrieval (MA-PaPSP) to PaPSP | PaPSP · Pets classification | AURC: 0.169 -> 0.076 | Improved |
| add memory retrieval (MA-PaPSP) to PaPSP | PaPSP · MS-COCO captioning | AURC Cider-N: 0.136 -> 0.126 | Improved |
| add memory retrieval (MA-PaPSP) to PaPSP | PaPSP · Flickr-30K captioning | AURC Cider-N: 0.229 -> 0.229 | No clear effect |
| add memory retrieval (MA-PaPSP) to PaPSP | PaPSP · VL-checklist ITM | AURC: 0.204 -> 0.207 | Hurt |
| add memory retrieval (MA-PaPSP) to PaPSP | PaPSP · UCF-101 classification | AURC: 0.113 -> 0.078 | Improved |
| fine-tune SP-VLM on in-domain data for PaPSP | PaPSP · Pets classification | AURC: 0.169 -> 0.074 | Improved |
| fine-tune SP-VLM on in-domain data for MA-PaPSP | MA-PaPSP · MS-COCO captioning | AURC Meteor: 0.288 -> 0.234 | Improved |
| fine-tune SP-VLM on in-domain data for MA-PaPSP | MA-PaPSP · Foil ITM | AURC: 0.167 -> 0.162 | Hurt |
| add proxy embedding and contrastive score to PaPSP (ablation) | PaPSP · captioning, ITM, classification | AURC | Improved |
| use soft nearest neighbor proxy (K>1) instead of top-1 retrieval (ablation) | MA-PaPSP · captioning, ITM, classification | AURC | Improved |
| increase neighborhood size K beyond about 15 (ablation) | MA-PaPSP · captioning, ITM, classification | AURC | Hurt |
| use image-to-text retrieval (i2tr) instead of other retrieval variants (ablation) | MA-PaPSP · captioning, ITM, classification | AURC | Improved |
Closest papers by meaning
- SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring (NeurIPS 2026 · similarity 0.89)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.89)
- ProAPO: Progressively Automatic Prompt Optimization for Visual Classification (CVPR 2025 · similarity 0.88)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
- Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP (AAAI 2025 · similarity 0.87)
- Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
