Research map / Vision-language and multimodal / Cross-modal learning

Adaptive Multimodal Fusion: Dynamic Attention Allocation for Intent Recognition

AAAI 2025 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MVCL-DAF, a multimodal intent recognition model with a dynamic attention allocation fusion module and multi-view contrastive learning anchored on text, evaluated on MIntRec and MIntRec 2.0.

Problem

Multimodal intent recognition where the informativeness of text, audio and visual modalities varies across modalities and samples, and existing fusion methods do not adapt attention weights per sample.

Approach

A Dynamic Attention Allocation Fusion (DAF) module with depth-adaptive gated networks computes per-sample intra-modal attention matrices for visual and acoustic features and inter-modal attention scores, while text is kept intact; multi-view InfoNCE contrastive learning uses text-plus-label anchors across text, visual, acoustic and fusion views.

Key results

• On MIntRec, MVCL-DAF reaches 74.72% ACC and 74.61% WF1, improving over the best baseline TCL-MAP by 1.03 and 1.23 points.
• On MIntRec 2.0, MVCL-DAF reaches 57.80% ACC and 55.05% WF1, improving over the best baseline by 0.81 and 0.72 points.
• In cross-dataset generalization, training on MIntRec 2.0 and testing on MIntRec gives 71.68% ACC versus 71.06% for MAG-BERT.

Measured effects reported in the paper

Surprising findings

• The fusion view alone performs close to the full multi-view model and even surpasses it on some metrics.
• Training on MIntRec and testing on MIntRec 2.0 gives MVCL-DAF no advantage in ACC (61.95 vs 62.19 for MAG-BERT).

Limitations

• Gains over baselines are modest (around 1 point), and results are averaged over 10 seeds without reported variance.
• Visual and acoustic features are taken as pre-extracted embeddings from the datasets, so raw-data processing is not part of the method.
• The method is evaluated only on two intent recognition datasets from the same TV-series source.

Experimental setup

Datasets: MIntRec MIntRec 2.0

Benchmarks: MIntRec MIntRec 2.0

Metrics: ACC WF1 WP R

Baselines: MulT MAG-BERT TCL-MAP

Builds on: MAG-BERT TCL-MAP DynMM FACTORCL InfoNCE

Closest papers by meaning

• Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition (AAAI 2024 · similarity 0.89)
• Contextual Augmented Global Contrast for Multimodal Intent Recognition (CVPR 2024 · similarity 0.88)
• MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding (ICML 2025 · similarity 0.88)
• MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations (ICLR 2024 · similarity 0.86)
• Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.85)
• Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.85)