Research map / Vision-language and multimodal / Cross-modal learning
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
Paul Pu Liang, Zihao Deng, Martin Q. Ma, James Zou, Louis-Philippe Morency, Russ Salakhutdinov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove factorization (single Z per modality) (ablation) | FactorCL-SUP · MIMIC | accuracy: 76.79% -> 68.16% | Hurt |
| remove factorization (single Z per modality) (ablation) | FactorCL-SSL · MIMIC | accuracy: 67.34% -> 65.22% | Mixed |
| remove factorization (ablation) | FactorCL-SUP · MOSI | accuracy: 70.69% -> 65.32% | Hurt |
| add MI upper-bound removal of task-irrelevant info (vs SimCLR) (ablation) | FactorCL-SSL · MOSEI | accuracy: 71.03% -> 74.88% | Improved |
| add MI upper-bound removal of task-irrelevant info (vs SupCon) (ablation) | FactorCL-SUP · MOSI | accuracy: 47.23% -> 70.69% | Improved |
| use multimodal augmentation instead of independent unimodal augmentations (ablation) | FactorCL-SSL · IRFL | accuracy: 92.77% -> 95.18% | Improved |
| use only shared representation instead of full (ablation) | FactorCL-SUP · MIMIC | accuracy: 76.79% -> 63.77% | Hurt |
| use only U1 representation instead of full (ablation) | FactorCL-SUP · MUStARD | accuracy: 69.86% -> 59.42% | Hurt |
| use only shared representation instead of full (ablation) | FactorCL-SUP · MOSEI | accuracy: 77.34% -> 77.17% | No clear effect |
| add unimodal CL term (Cross+Self vs SimCLR) | SimCLR · MIMIC | accuracy: 66.67% -> 65.20% | Hurt |
| replace standard CL with FactorCL | CLIP continued pretraining · IRFL | accuracy: 96.39% -> 98.80% | Improved |
Closest papers by meaning
- What to align in multimodal contrastive learning? (ICLR 2025 · similarity 0.90)
- Continual Multimodal Contrastive Learning (NeurIPS 2025 · similarity 0.90)
- QUEST: Quadruple Multimodal Contrastive Learning with Constraints and Self-Penalization (NeurIPS 2024 · similarity 0.89)
- On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.88)
- ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining (NeurIPS 2026 · similarity 0.87)
- Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
