Research map / Vision-language and multimodal / Cross-modal learning
Boosting Multimodal Learning via Disentangled Gradient Learning
Shicai Wei, Chunbo Luo, Yang Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multimodal gradient truncation (MT) (ablation) | concat multimodal baseline · CREMA-D | accuracy: 65.10 -> 73.31 | Improved |
| add unimodal gradient truncation (UT) (ablation) | concat multimodal baseline · CREMA-D | accuracy: 65.10 -> 74.22 | Improved |
| combine MT and UT (full DGL) (ablation) | concat multimodal baseline · CREMA-D | accuracy: 65.10 -> 77.48 | Improved |
| raise unimodal loss weight alpha from 1 to 4 (ablation) | DGL · CREMA-D | accuracy: 73.35 -> 77.48 | Improved |
| raise alpha from 4 to 5 (ablation) | DGL · CREMA-D | accuracy: 77.48 -> 77.08 | Hurt |
| raise alpha from 1 to 4 (ablation) | G-Blending · CREMA-D | accuracy: 67.89 -> 69.21 | Improved |
| replace concat baseline with DGL | Concat · VGGSound | multimodal accuracy: 47.90 -> 52.53 | Improved |
| replace concat baseline with DGL | Concat · Kinetics-Sounds | multimodal accuracy: 64.45 -> 74.78 | Improved |
| apply DGL to MMTM fusion | MMTM multimodal baseline · CREMA-D | multimodal accuracy: 60.12 -> 75.00 | Improved |
| apply DGL to three-modality MLP fusion | Multimodal baseline · MOSI | multimodal accuracy: 76.83 -> 79.78 | Improved |
| apply DGL to ESANet (from scratch) | ESANet · NYUv2 | mIoU: 38.59 -> 41.67 | Improved |
| apply DGL to ESANet (ImageNet pretrained) | ESANet · NYUv2 | mIoU: 48.48 -> 50.10 | Improved |
Closest papers by meaning
- Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning (NeurIPS 2025 · similarity 0.91)
- Beyond Modality Fusion: Deep Ensembles for Multimodal Classification (NeurIPS 2026 · similarity 0.90)
- Improving Multimodal Learning Balance and Sufficiency through Data Remixing (ICML 2025 · similarity 0.90)
- MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance (ICML 2024 · similarity 0.90)
- Asymmetric Reinforcing Against Multi-Modal Representation Bias (AAAI 2025 · similarity 0.90)
- MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
