Research map / Vision-language and multimodal / Cross-modal learning

Multimodal Negative Learning

NeurIPS 2025 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Multimodal Negative Learning (MNL), where the dominant modality guides the weak modality to suppress non-target classes (with dynamic RDM/IM selection via UCoM), added to late fusion methods to improve robustness under noise and modality imbalance.

Problem

Modality imbalance in multimodal late fusion, where weak modalities are forced to align with dominant ones (positive learning), suppressing their unique information and reducing robustness under noise.

Approach

Instead of pushing weak modalities to raise target-class predictions, the robust dominant modality guides the inferior modality to suppress non-target classes (negative learning), with the guidance direction chosen dynamically by the Unimodal Confidence Margin; theory links a larger UCoM to a tighter robustness lower bound.</1>

Key results

• On MVSA at Gaussian noise level 10, LF+MNL reaches 63.01% vs 55.16% for static late fusion; PDF+MNL reaches 63.78% vs 63.09%.
• On CREMA-D (Gaussian, eps=0), LF+MNL improves from 68.04% to 73.71%, a +5.67 gain.
• On MathQA, LATE FUSION+MNL improves fusion accuracy from 50.89% to 51.42%; on Visual7W VQA, from 84.87% to 85.21%.

Measured effects reported in the paper

Surprising findings

• Gains from MNL are smaller when applied to dynamic fusion methods, which may down-weight the weaker modality and misalign with MNL.
• Non-target guidance yields higher KL divergence between modality predictions yet better multimodal accuracy than all-class guidance.

Limitations

• Gains are smaller for dynamic fusion methods, which can down-weight weak modalities.
• Gains are modest on NYU Depth V2 where modality imbalance is minimal.
• Guidance direction depends on the UCoM and confidence heuristic (Definition 3.3); no general guarantee beyond the analysed cases.

Experimental setup

Datasets: UMPC Food-101 MVSA NYU Depth V2 CREMA-D CMU-MOSEI MathQA Visual7W

Benchmarks: MVSA UMPC Food-101 NYU Depth V2 CREMA-D CMU-MOSEI MathQA Visual7W

Metrics: accuracy

Baselines: Late Fusion DynMM TMC QMF PDF Deep CCA LFM SUFA

Builds on: late fusion Quality-aware Multimodal Fusion (QMF) PDF Lipschitz robustness bounds

Closest papers by meaning

• Asymmetric Reinforcing Against Multi-Modal Representation Bias (AAAI 2025 · similarity 0.91)
• Rethinking Multimodal Learning from the Perspective of Mitigating Classification Ability Disproportion (NeurIPS 2025 · similarity 0.90)
• Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning (NeurIPS 2025 · similarity 0.88)
• Facilitating Multimodal Classification via Dynamically Learning Modality Gap (NeurIPS 2024 · similarity 0.88)
• MIDAS: Misalignment-based Data Augmentation Strategy for Imbalanced Multimodal Learning (NeurIPS 2025 · similarity 0.87)
• Improving Multimodal Learning Balance and Sufficiency through Data Remixing (ICML 2025 · similarity 0.87)