Research map / Vision-language and multimodal / Cross-modal learning
SceneTrilogy: On Human Scene-Sketch and Its Complementarity With Photo and Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add modality-specific/agnostic disentanglement with τₖ (ablation) | SceneTrilogy · FS-COCO FG-STBIR | Acc.@1: +0.4 | Helped |
| Add cross-attention to model interactions between query modalities (ablation) | SceneTrilogy · FS-COCO FG-STBIR | Acc.@1: +0.6 | Helped |
| Use contrastive loss instead of simple triplet loss (ablation) | SceneTrilogy · FS-COCO FG-STBIR | Acc.@1: +0.2 | Helped |
| Add sketch as a guiding signal for image captioning | Image captioning · Subjective captioning pilot study | B-4: +10.9 | Helped |
| Add sketch as a guiding signal for image captioning | Image captioning · Subjective captioning pilot study | CIDEr: +16.1 | Helped |
| Use element-wise addition to fuse sketch and text | SCM · SketchyCOCO FG-STBIR | Acc.@1: 38.5 vs. 39.1 | Hurt |
| Replace naive addition with a weighted sum | QST · SketchyCOCO FG-STBIR | Acc.@1: 38.9 vs. 38.5 | Helped |
| Use CLIP-LN rather than SceneTrilogy | Text-based image retrieval · SketchyCOCO FG-TBIR | Acc.@1: 21.5 vs. 22.1 | Helped |
| Use CLIP-LN rather than SceneTrilogy | Text-based image retrieval · FS-COCO FG-TBIR | Acc.@1: 13.7 vs. 14.8 | Helped |
Closest papers by meaning
- Open Vocabulary Semantic Scene Sketch Understanding (CVPR 2024 · similarity 0.89)
- The Scene Language: Representing Scenes with Programs, Words, and Embeddings (CVPR 2025 · similarity 0.88)
- O3SLM: Open Weight, Open Data, and Open Vocabulary Sketch-Language Model (AAAI 2026 · similarity 0.87)
- CrossOver: 3D Scene Cross-Modal Alignment (CVPR 2025 · similarity 0.87)
- Stitch and Tell: A Structured Data Augmentation Method for Spatial Understanding (NeurIPS 2025 · similarity 0.87)
- Object-centric binding in Contrastive Language-Image Pretraining (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.