Research map / Vision-language and multimodal / Cross-modal learning
Learning Multi-Modal Word Representation Grounded in Visual Context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly train shared word embeddings with surrounding-object visual contexts and text | Text-only skip-gram · MEN | Spearman correlation: 69 -> 75 | Helped |
| Jointly train shared word embeddings with surrounding-object visual contexts and text | Text-only skip-gram · WordSim353 | Spearman correlation: 63 -> 67 | Helped |
| Use joint multimodal training instead of sequential concatenation and PCA | Object-context and text embeddings · Concreteness prediction | R²: 43.7 -> 42.9 | Helped |
| Use surrounding objects rather than image patches as visual contexts (ablation) | Visual-context model · MEN | Spearman correlation: 35 -> 64 | Helped |
| Add low-level spatial features by concatenation to object contexts (ablation) | O visual-context model · MEN | Spearman correlation: 64 -> 58 | No clear effect |
| Add high-level spatial features through bilinear interaction with object contexts (ablation) | O visual-context model · MEN | Spearman correlation: 64 -> 62 | No clear effect |
| Combine direct visual object features with visual contexts | L visual-feature model and O visual-context model · MEN | Spearman correlation: 64 -> 66 | Helped |
| Add high-level spatial features with bilinear integration (ablation) | O + T · SimLex-999 | Spearman correlation: 35 -> 34 | Helped |
| Add spatial context to visual contexts (ablation) | Multimodal context-driven model · Evaluation tasks overall | Performance: 6% improv. | Helped |
| Use visual context rather than direct visual object features | Multimodal model · Evaluation tasks overall | Performance: 3.2% improvement | Helped |
| Use high-level object contexts rather than low-level image contexts (ablation) | Visual-context model · RQ1 evaluation tasks | Performance: +31% | Helped |
| Combine visual object appearance with visual context | L + O · Feature-norm prediction: Color | F1: 41 -> 27 | Hurt |
Closest papers by meaning
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.90)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.89)
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.88)
- What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models (CVPR 2026 · similarity 0.88)
- Remodeling Semantic Relationships in Vision-Language Fine-Tuning (AAAI 2026 · similarity 0.88)
- A More Word-like Image Tokenization for MLLMs (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.