Research map / Vision-language and multimodal / Human and embodied modeling
Imagined Visual Representations as Multimodal Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| concatenate imagined (mapped) visual vectors with GloVe | GloVe · SemSim ALL | Spearman correlation: 0.753 -> 0.783 | Helped |
| concatenate imagined visual vectors with GloVe | GloVe · 7 VIS regions | Spearman correlation: avg +4.6% (MAP-CNN), +2.8% (MAP-Clin) | Helped |
| concatenate original CNN visual vectors with GloVe (CONC) | GloVe · VIS regions | Spearman correlation | No clear effect |
| use imagined visual vectors instead of original visual vectors in concatenation | CONC · VIS regions | Spearman correlation: MAP-CNN better in 6 of 7 sets | Helped |
| concatenate with GloVe instead of imagined vectors alone (ablation) | MAP vectors · Wordsim353 ALL | Spearman correlation: 0.443 -> 0.687 | Helped |
| use imagined vectors alone instead of GloVe | GloVe · Simlex999 ALL | Spearman correlation: 0.408 -> 0.322 | Hurt |
| use neural-network instead of linear mapping (ablation) | MAP-C · SemSim ALL | Spearman correlation: 0.783 vs 0.785 | No clear effect |
| reduce hidden units from 300 to 50 (ablation) | neural network mapping · word similarity | Spearman correlation | Hurt |
| swap CNN (VGG-m-128 to AlexNet or ResNet) (ablation) | MAP-C · word similarity | Spearman correlation | No clear effect |
| swap GloVe for word2vec (ablation) | MAP-C · word similarity | Spearman correlation | Hurt |
| use CCA mapping instead of directional mapping (ablation) | GloVe · word similarity | Spearman correlation | Hurt |
| maxpool instead of average visual features (ablation) | MAP · word similarity | Spearman correlation | No clear effect |
Closest papers by meaning
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.90)
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.90)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.90)
- Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval (NeurIPS 2025 · similarity 0.89)
- JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.