Research map / Vision-language and multimodal / Cross-modal learning
Balanced Multimodal Learning via On-the-Fly Gradient Modulation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply OGM-GE to concatenation fusion | Audio-visual fusion model · CREMA-D | Acc: 51.7 -> 61.9 | Helped |
| Apply OGM-GE to concatenation fusion | Audio-visual fusion model · VGGSound | Acc: 49.1 -> 50.6 | Helped |
| Apply OGM-GE to concatenation fusion | Audio-visual fusion model · VGGSound | mAP: 52.5 -> 53.9 | Helped |
| Add Gaussian-noise GE to OGM (ablation) | Concatenation fusion with OGM · CREMA-D | Acc: 59.0 -> 61.9 | Helped |
| Add Gaussian-noise GE to OGM (ablation) | Concatenation fusion with OGM · Kinetics-Sounds | Acc: 61.1 -> 62.3 | Helped |
| Apply OGM-GE to the existing architecture | TBN · Kinetics-Sounds | Acc: 60.8 -> 63.1 | Helped |
| Apply OGM-GE to the existing architecture | PSP · AVE | Acc: 76.2 -> 76.9 | Helped |
| Apply OGM-GE while using Adam | Audio-visual fusion model · CREMA-D | Acc: 49.7 -> 54.6 | Helped |
| Increase learning rate (ablation) | Concatenation fusion · CREMA-D | Acc: 50.4 -> 51.0 -> 51.8 | Helped |
| Increase learning rate (ablation) | Concatenation fusion · VGGSound | Acc: 48.3 -> 48.7 -> 49.1 | Helped |
| Increase batch size (ablation) | Concatenation fusion with lr=1e-3 · CREMA-D | Acc: 51.8 -> 50.2 -> 48.6 | Hurt |
| Increase batch size (ablation) | Concatenation fusion with lr=1e-3 · VGGSound | Acc: 49.1 -> 48.8 -> 47.7 | Hurt |
Closest papers by meaning
- Mixtures of Experts for Audio-Visual Learning (NeurIPS 2024 · similarity 0.90)
- Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling (CVPR 2024 · similarity 0.88)
- Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning (NeurIPS 2025 · similarity 0.88)
- Improving Multimodal Learning Balance and Sufficiency through Data Remixing (ICML 2025 · similarity 0.88)
- Asymmetric Reinforcing Against Multi-Modal Representation Bias (AAAI 2025 · similarity 0.87)
- Asymmetric Hierarchical Anchoring for Robust Audio–Visual Cross-Modal Generalization (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.