Research map / Vision-language and multimodal / Human and embodied modeling
Who's Waldo? Linking People Across Text and Images
Yuqing Cui, Apoorv Khandelwal, Yoav Artzi, Noah Snavely, Hadar Averbuch-Elor
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add intra-image box-name matching loss (ablation) | UNITER-based model · Who's Waldo test | accuracy: 31.4 -> 63.5 | Improved |
| add inter-image matching loss (ablation) | UNITER-based model · Who's Waldo test | accuracy: 61.9 -> 63.5 | Improved |
| add unlinked box classification loss (ablation) | UNITER-based model · Who's Waldo test | accuracy: 61.7 -> 63.5 | Improved |
| start from pretrained UNITER weights (ablation) | UNITER-based model · Who's Waldo test | accuracy: 50.2 -> 63.5 | Improved |
| add visual features (ablation) | UNITER-based model · Who's Waldo test | accuracy: 55.4 -> 63.5 | Improved |
| add spatial features (ablation) | UNITER-based model · Who's Waldo test | accuracy: 58.0 -> 63.5 | Improved |
| add textual features (ablation) | UNITER-based model · Who's Waldo test | accuracy: 51.3 -> 63.5 | Improved |
| add optimal transport loss (ablation) | UNITER-based model · Who's Waldo test | accuracy: 63.5 -> 62.2 | No clear effect |
| use bipartite matching at inference instead of max box (ablation) | UNITER-based model · Who's Waldo test | accuracy: 63.5 -> 61.9 | Hurt |
| add horizontal flip augmentation (ablation) | UNITER-based model · Who's Waldo test | accuracy: 63.5 -> 53.8 | Hurt |
| add box translation augmentation (ablation) | UNITER-based model · Who's Waldo test | accuracy: 63.5 -> 62.0 | No clear effect |
| add color jittering augmentation (ablation) | UNITER-based model · Who's Waldo test | accuracy: 63.5 -> 63.0 | No clear effect |
Closest papers by meaning
- Linking People across Text and Images Based on Social Relation Reasoning (AAAI 2023 · similarity 0.86)
- Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions (CVPR 2024 · similarity 0.86)
- A Large-Scale Human-Centric Benchmark for Referring Expression Comprehension in the LMM Era (NeurIPS 2024 · similarity 0.84)
- PLIP: Language-Image Pre-training for Person Representation Learning (NeurIPS 2024 · similarity 0.84)
- Behavioral Geometric Supervision Aligns Video Foundation Models with Human Social Perception (NeurIPS 2026 · similarity 0.84)
- A Pedestrian is Worth One Prompt: Towards Language Guidance Person Re-Identification (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
