Research map / Vision-language and multimodal / Cross-modal learning
Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks
Yang Yang, Hongpeng Pan, Qing-Yuan Jiang, Yi Xu, Jinhui Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AMSS+ rebalancing over concatenation fusion | Concat · Kinetics-Sound | accuracy: 64.55 -> 72.25 | Improved |
| add AMSS+ rebalancing over concatenation fusion | Concat · CREMA-D | accuracy: 63.31 -> 70.30 | Improved |
| use unbiased AMSS+ mask instead of biased AMSS | AMSS · Kinetics-Sound | accuracy: 68.96 -> 72.25 | Improved |
| use unbiased AMSS+ mask instead of biased AMSS | AMSS · Twitter-15 | accuracy: 75.89 -> 75.12 | Hurt |
| use non-uniform adaptive sampling instead of uniform random sampling (ablation) | AMSS+ · Kinetics-Sound | accuracy: 66.45 -> 72.25 | Improved |
| use non-uniform adaptive sampling instead of uniform random sampling (ablation) | AMSS+ · CREMA-D | accuracy: 65.59 -> 70.16 | Improved |
| add backbone mask in addition to classifier mask (ablation) | AMSS · Kinetics-Sound | accuracy: 65.37 -> 68.96 | Improved |
| add classifier mask in addition to backbone mask (ablation) | AMSS+ · Kinetics-Sound | accuracy: 69.15 -> 72.25 | Improved |
| set tau below 1 (0.25) instead of 1 (ablation) | AMSS+ · Kinetics-Sound | accuracy: 70.20 -> 72.25 | Improved |
| set tau very small (0.10) instead of 0.20 (ablation) | AMSS+ · Kinetics-Sound | accuracy: 72.52 -> 72.17 | Hurt |
| scale with 1/(p_j+L) without masking (rho=1) (ablation) | AMSS+ · Kinetics-Sound | accuracy: 72.25 -> 69.39 | Hurt |
| apply AMSS+ to Affine fusion | Affine · Kinetics-Sound | accuracy: 64.24 -> 69.08 | Improved |
Closest papers by meaning
- ReconBoost: Boosting Can Achieve Modality Reconcilement (ICML 2024 · similarity 0.86)
- Improving Multimodal Learning Balance and Sufficiency through Data Remixing (ICML 2025 · similarity 0.86)
- RedCore: Relative Advantage Aware Cross-Modal Representation Learning for Missing Modalities with Imbalanced Missing Rates (AAAI 2024 · similarity 0.86)
- Quantifying and Enhancing Multi-modal Robustness with Modality Preference (ICLR 2024 · similarity 0.85)
- Modalities Contribute Unequally: Enhancing Medical Multi-modal Learning through Adaptive Modality Token Re-balancing (ICML 2025 · similarity 0.85)
- An Optimal Transport-based Latent Mixer for Robust Multi-modal Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
