Research map / Vision-language and multimodal / Cross-modal learning
Learning Multimodal Word Representation via Dynamic Fusion Methods
Shaonan Wang, Jiajun Zhang, Chengqing Zong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dynamic fusion with vector gates instead of Ridge | Multimodal word representations · VIS and ZS regions across six benchmarks | Spearman correlation | Improved |
| Use vector gates rather than value gates (ablation) | M-gate, C-gate, and S-gate · Six evaluation benchmarks | Spearman correlation | Improved |
| Use category-specific vector gating instead of modality- or sample-specific vector gating (ablation) | Dynamic fusion · Six evaluation benchmarks | Spearman correlation | Hurt |
| Reduce training data for vector-gate models (ablation) | Dynamic fusion with vector gates · All evaluation benchmarks | average Spearman correlation | Hurt |
| Reduce training data for value-gate models (ablation) | Dynamic fusion with value gates · All evaluation benchmarks | average Spearman correlation | No clear effect |
| Concatenate original visual vectors with linguistic vectors instead of using predicted visual vectors | CONC · Four of six benchmarks | Spearman correlation | Hurt |
| Use predicted visual vectors with Mapping or Ridge instead of linguistic vectors alone | Mapping and Ridge · Five of six benchmarks, ALL and VIS | Spearman correlation | Improved |
| Use Mapping or Ridge instead of Glove alone | Mapping and Ridge · ZS region | Spearman correlation | Mixed |
| Use vector gates rather than value gates (ablation) | Dynamic fusion models · All six evaluation benchmarks | Spearman correlation | Improved |
| Use S-gate-val (ablation) | Sample-specific fusion · USF concreteness words | Spearman correlation (concreteness vs. linguistic-to-visual weight ratio): 0.614 | Improved |
| Use S-gate-vec (ablation) | Sample-specific fusion · USF concreteness words | Spearman correlation (concreteness vs. linguistic-to-visual weight ratio): 0.458 | Improved |
Closest papers by meaning
- A Concept-Based Explainability Framework for Large Multimodal Models (NeurIPS 2024 · similarity 0.86)
- Fusion or Confusion? Multimodal Complexity Is Not All You Need (NeurIPS 2026 · similarity 0.85)
- Beyond Modality Fusion: Deep Ensembles for Multimodal Classification (NeurIPS 2026 · similarity 0.85)
- Debiased Multimodal Understanding for Human Language Sequences (AAAI 2025 · similarity 0.85)
- Amplifying Prominent Representations in Multimodal Learning via Variational Dirichlet Process (NeurIPS 2025 · similarity 0.85)
- Facilitating Multimodal Classification via Dynamically Learning Modality Gap (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
