Research map / Vision-language and multimodal / Human and embodied modeling
ILIAS: Instance-Level Image retrieval At Scale
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ILIAS, a large-scale, multi-domain instance-level image and text-to-image retrieval benchmark with 1,000 objects and 100M YFCC100M distractors, and benchmarks foundation models with linear adaptation and re-ranking.
Problem
Existing instance-level image retrieval datasets are small, narrow in domain, noisy in ground truth, or saturated, so they cannot evaluate foundation models on recognizing particular objects at scale.
Approach
Build query objects manually and verify they emerged after 2014, so that a 100M-image YFCC100M (crawled 2014) distractor set contains no false negatives without extra annotation; evaluate global, re-ranking, linear-adapted, and text-to-image retrieval.
Key results
- Best global image-to-image model SigLIP2 ViT-L reaches 37.3 mAP@1k on the 100M set without adaptation... with linear adaptation SigLIP ViT-L reaches 28.9 mAP@1k on ILIAS and 34.3 on mini-ILIAS.
- Re-ranking with AMES local descriptors on linearly adapted SigLIP raises mAP@1k from 28.9 to 35.6, the best result among the reported methods.
- Text-to-image SigLIP2 achieves 31.3 mAP@1k vs image-to-image 37.3 in the no-adaptation setting... reported gap is small; SigLIP text-to-image 18.1 vs image-to-image 19.6 mAP@1k on the 100M set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add linear adaptation trained on UnED (ablation) | SigLIP ViT-L image-to-image retrieval · ILIAS 100M | mAP@1k: 19.6 -> 28.9 | Helped |
| add linear adaptation trained on UnED (ablation) | SigLIP2 ViT-L image-to-image retrieval · ILIAS 100M | mAP@1k: 20.8 -> 31.3 | Helped |
| rerank with AMES using local descriptors | SigLIP ViT-L image-to-image retrieval · ILIAS 100M | mAP@1k: 19.6 -> 26.4 | Helped |
| rerank with AMES using local descriptors | SigLIP ViT-L with linear adaptation · ILIAS 100M | mAP@1k: 28.9 -> 35.6 | Helped |
| apply global query expansion with α=1 | SigLIP ViT-L with linear adaptation · ILIAS 100M | mAP@1k: 28.9 -> 33.7 | Helped |
| increase query expansion setting from αQE1 to αQE2 (ablation) | SigLIP ViT-L with linear adaptation · ILIAS 100M | mAP@1k: 33.7 -> 31.5 | Hurt |
| increase query expansion setting from αQE1 to αQE5 (ablation) | SigLIP ViT-L with linear adaptation · ILIAS 100M | mAP@1k: 33.7 -> 23.5 | Hurt |
| use detailed text queries instead of image queries | SigLIP ViT-L retrieval · ILIAS 100M | mAP@1k: 19.6 vs 18.1 | Mixed |
| use a challenging VLM-selected 5M distractor subset instead of a random 5M subset (ablation) | SigLIP with linear adaptation · mini-ILIAS 5M | mAP@1k: 34.3 vs 41.8±0.1 | Hurt |
| use 5M-mini distractors rather than 100M distractors (ablation) | SigLIP with linear adaptation · ILIAS distractor-set comparison | mAP@1k: 28.9 -> 34.3 | Helped |
| use 5M-mini distractors rather than 100M distractors (ablation) | DINOv2 with linear adaptation · ILIAS distractor-set comparison | mAP@1k: 15.3 -> 18.8 | Helped |
| use a higher input resolution (ablation) | SigLIP ViT-B · ILIAS 100M | mAP@1k: 11.5 -> 15.6 -> 16.6 | Helped |
Surprising findings
- Smaller ViT-B models at large resolution (512/724) outperform larger ViT-L models at small resolution (256/384) for SigLIP.
- Text-to-image retrieval with VLMs is surprisingly close to image-to-image retrieval, despite text queries being detailed descriptions.
Limitations
- Local descriptor re-ranking is memory and computationally costly at ILIAS scale.
- Dealing with small objects and multi-object clutter remains a major weakness; oracle results show large headroom.
- Some categories are hurt by AMES re-ranking because it is trained on landmarks.
Experimental setup
Datasets: UnED YFCC100M GLDv2 SOP INSTRE
Benchmarks: ILIAS mini-ILIAS (5M) INSTRE GLDv2 SOP
Metrics: mAP@1k oracle re-ranking mAP@1k
Baselines: SigLIP SigLIP2 DINOv2 OpenCLIP EVA-CLIP MetaCLIP OAI-CLIP AMES αQE Chamfer Similarity Spatial Verification
Builds on: UnED AMES YFCC100M GLDv2 αQE
Closest papers by meaning
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.90)
- Find your Needle: Small Object Image Retrieval via Multi-Object Attention Optimization (NeurIPS 2025 · similarity 0.87)
- MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions (ICML 2024 · similarity 0.87)
- INQUIRE: A Natural World Text-to-Image Retrieval Benchmark (NeurIPS 2024 · similarity 0.86)
- QuARI: Query Adaptive Retrieval Improvement (NeurIPS 2025 · similarity 0.85)
- Context-I2W: Mapping Images to Context-Dependent Words for Accurate Zero-Shot Composed Image Retrieval (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.