Research map / Vision-language and multimodal / Human and embodied modeling
Sherlock: Scalable Fact Learning in Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use separate subject and predicate-object image encoder branches (Model 2) rather than the shared structured encoder (Model 1) | Sherlock · Large-scale fact retrieval | language retrieval K10: 18.000 -> 20.300 | Helped |
| Use separate subject and predicate-object image encoder branches (Model 2) rather than the shared structured encoder (Model 1) | Sherlock · Pascal Actions | language retrieval top 1: 74.080 -> 74.760 | Helped |
| Use separate subject and predicate-object image encoder branches (Model 2) rather than the shared structured encoder (Model 1) | Sherlock · Stanford40 | visual retrieval mAP: 74.57 -> 73 | Hurt |
| Train image-caption similarity models on image-fact pairs instead of using MS COCO pretrained weights | Image-Sentence TACL15 · Stanford40 | language retrieval top 1: 33.73 -> 60.86 | Helped |
| Train image-caption similarity models on image-fact pairs instead of using MS COCO pretrained weights | Image-Sentence TACL15 · 6DS | language retrieval top 1: 15.71 -> 26.13 | Helped |
| Relate subject, predicate, and object in separate structured embedding dimensions rather than averaging their embeddings (ablation) | Sherlock Model 2 · Large-scale fact retrieval | retrieval performance | Mixed |
Closest papers by meaning
- Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.85)
- Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-Modal Structured Representations (AAAI 2024 · similarity 0.84)
- Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models (ICLR 2025 · similarity 0.84)
- Chain-of-Thought Guided Multi-Modal Object Re-Identification (CVPR 2026 · similarity 0.84)
- ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.84)
- Will Multimodal Models Be Dazzled by Multi-Image Visual Puzzles? (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.