Research map / Vision-language and multimodal / Cross-modal learning
Towards Uniformity and Alignment for Multimodal Representation Learning
Wenzhe Yin, Pan Zhou, Zehao Xiao, Jie Liu, Shujian Yu, Jan-Jakob Sonke, Stratis Gavves
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add tuple-level uniformity U(C) (ablation) | UniAlign · MSR-VTT ablation | avg Recall@1: 36.6 -> 38.6 | Improved |
| add tuple-level alignment L_vol (ablation) | UniAlign · MSR-VTT ablation | avg Recall@1: 36.6 -> 37.6 | Improved |
| add both U(C) and L_vol (ablation) | UniAlign · MSR-VTT ablation | avg Recall@1: 36.6 -> 41.8 | Improved |
| add L_vol on top of U(C) (ablation) | UniAlign · MSR-VTT ablation | avg Recall@1: 38.6 -> 41.8 | Improved |
| add tuple-level losses U(C) and L_vol (ablation) | UniAlign · Zero-shot retrieval average | T2V Recall@1: 57.4 -> 58.8 | Improved |
| add tuple-level losses U(C) and L_vol (ablation) | UniAlign · DiDeMo zero-shot | V2T Recall@1: 51.9 -> 51.6 | Hurt |
| change centroid temperature from 0.01 to 0.07 (ablation) | UniAlign · MSR-VTT | avg Recall@1: 55.2 -> 55.3 | No clear effect |
| increase lambda_align from 0.5 to 1.5 (ablation) | UniAlign basic instantiation · MSR-VTT (from scratch) | avg Recall@1: 27.8 -> 19.45 | Hurt |
| increase lambda_align from 0.2 to 0.5 (ablation) | UniAlign basic instantiation · MSR-VTT (from scratch) | avg Recall@1: 25.3 -> 27.8 | Improved |
| replace GRAM with UniAlign | GRAM · Zero-shot retrieval | T2V Recall@1: +3.0 R@1 on T2V and 1.6 on V2T | Improved |
| replace VAST with UniAlign from same pretrained weights | VAST · Zero-shot retrieval | Recall@1: 8.7 R@1 on T2V and 6.4 R@1 on V2T | Improved |
| replace GRAM with UniAlign (Euclidean) | GRAM · VGGSound generation (Kandinsky) | FID: 87.23 -> 46.60 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- VAST — VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset (NeurIPS 2023) · better on 2 of 2 shared results
- ImageBind — ImageBind: One Embedding Space To Bind Them All (CVPR 2023) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Aligning Multimodal Representations through an Information Bottleneck (ICML 2025 · similarity 0.92)
- With Limited Data for Multimodal Alignment, Let the STRUCTURE Guide You (NeurIPS 2025 · similarity 0.91)
- On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.90)
- Understanding the Emergence of Multimodal Representation Alignment (ICML 2025 · similarity 0.90)
- Gramian Multimodal Representation Learning and Alignment (ICLR 2025 · similarity 0.90)
- Scaling Language-centric Omnimodal Representation Learning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
