Research map / 3D vision / 3D generation
Where am I? Scene Retrieval with Language
Jiaqi Chen, Daniel Barath, Iro Armeni, Marc Pollefeys, Hermann Blum
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| score matches by embedding cosine similarity instead of matching probability (ablation) | Text2SGM · ScanScribe (top-1 of 10) | recall: 63.70 -> 68.27 | Improved |
| precompute scene graph embeddings with a fixed paired input (retrieval-based matching) (ablation) | Text2SGM · ScanScribe (top-1 of 10) | recall: 68.27 -> 68.61 | No clear effect |
| train with both cosine similarity and matching probability losses instead of cosine only (ablation) | Text2SGM · Human (top-1 of 10) | recall: 44.88 -> 51.38 | Improved |
| train with both losses instead of cosine only (ablation) | Text2SGM · ScanScribe (top-1 of 10) | recall: 65.17 -> 62.70 | No clear effect |
| use more self/cross-attention modules (N=1 to N=4) (ablation) | Text2SGM · ScanScribe (top-1 of 10) | recall: 62.70 -> 23.02 | Hurt |
| use more self/cross-attention modules (N=1 to N=2) (ablation) | Text2SGM · Human (top-1 of 10) | recall: 51.38 -> 45.05 | Hurt |
| use Text2SGM scene graph embeddings instead of CLIP image embeddings on open-set human queries (top-5 of 10) | Text2SGM · Human (top-5 of 10) | recall: 95.11 (CLIP2CLIP) vs 94.87 | Mixed |
| use scene graphs and text-graphs instead of CLIP on ScanScribe | Text2SGM · ScanScribe (top-1 of 10) | recall: 33.09 -> 68.27 | Improved |
| use point-cloud based Text2Pos coarse localization indoors | Text2Pos · ScanScribe, 55 scenes (top-5 of all) | recall: 10.27 vs 76.34 for Text2SGM | Hurt |
| evaluate with GPT-4 queries generated from images (biased toward CLIP) | Text2SGM · ScanScribe (top-1 of 10) | recall: 33.07 -> 34.22 | No clear effect |
Closest papers by meaning
- Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers (NeurIPS 2024 · similarity 0.88)
- The Scene Language: Representing Scenes with Programs, Words, and Embeddings (CVPR 2025 · similarity 0.88)
- ESCA: Contextualizing Embodied Agents via Scene-Graph Generation (NeurIPS 2025 · similarity 0.86)
- Scene Graph Generation with Role-Playing Large Language Models (NeurIPS 2024 · similarity 0.86)
- Multi-modal Situated Reasoning in 3D Scenes (NeurIPS 2024 · similarity 0.86)
- ASHiTA: Automatic Scene-grounded HIerarchical Task Analysis (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
