Research map / Vision-language and multimodal / Multimodal LLMs

Evolutionary Multimodal Reasoning via Hierarchical Semantic Representation for Intent Recognition

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HIER, which builds a three-level semantic hierarchy (label-guided concept clustering, JS-divergence relation selection) and feeds it into Qwen2-VL with structured CoT and a logit-based self-evolution refinement for multimodal intent recognition.

Problem

Multimodal intent recognition struggles to model hierarchical semantics of complex intents and lacks self-evolving reasoning over multimodal representations.

Approach

Organizes multimodal tokens into label-guided Spherical K-Means++ concepts and JS-divergence-selected inter-concept relations, injects them into an MLLM via a three-stage CoT, and refines features using Yes/No logit confidence scores from the MLLM's own generation head.

Key results

• On MIntRec, HIER reaches 80.00 ACC and 76.91 F1, versus 76.56 ACC and 74.59 F1 for Qwen2-VL.
• On MIntRec2.0, HIER reaches 64.15 ACC and 60.31 F1, versus 60.66 ACC for the best baseline VideoLLaMA2 (60.11) and 59.82 for Qwen2-VL.
• Applied to LLaVA-NeXT on MIntRec, HIER improves ACC from 72.65 to 78.20 and F1 from 64.94 to 75.33.

Surprising findings

• Removing the concept module leaves MIntRec performance nearly unchanged (within about 0.3-1% of full HIER), attributed to the small dataset size.
• Qwen2-VL alone outperforms several specialized multimodal intent recognition baselines on MIntRec, and on MIntRec2.0 its F1 (47.73) is lower than several baselines.

Limitations

• Performance lags human level on categories such as Taunt, Flaunt, and Leave, attributed to semantic inconsistencies across modalities.
• Gains on MELD-DA are modest relative to the strongest baselines (often under 2%).

Experimental setup

Datasets: MIntRec MIntRec2.0 MELD-DA

Benchmarks: MIntRec MIntRec2.0 MELD-DA

Metrics: ACC F1 P R WF1 WP

Baselines: MISA MAG-BERT MulT TCL-MAP SDIF-DA MIntOOD MVCL-DAF Qwen2-VL LLaVA-NeXT VideoLLaMA2

Builds on: Qwen2-VL Spherical K-Means++ LoRA MIntRec TCL-MAP

Closest papers by meaning

• Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning (ICLR 2026 · similarity 0.87)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.87)
• Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.87)
• MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations (ICLR 2024 · similarity 0.86)
• Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark (ICML 2025 · similarity 0.86)
• GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking (CVPR 2026 · similarity 0.86)