Research map / Vision-language and multimodal / Human and embodied modeling
McHirc: A Multimodal Benchmark for Chinese Idiom Reading Comprehension
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MChIRC, a 44,433 image-text pair benchmark for multimodal Chinese idiom cloze, and proposes DCIGN, a dual-contrastive graph network that reaches 73.00% average accuracy.
Problem
Chinese idiom reading comprehension (cloze selection among seven candidate idioms) has been studied only with text, although idioms often appear with images in real scenarios with short contexts.
Approach
Combines coarse and fine dual-contrastive text-image alignment, a GCN over candidate idiom embeddings, and cross-attention fusion to select the idiom, on a new multimodal idiom dataset with short sentences.
Key results
- DCIGN reaches 73.00% average accuracy across Dev, Test, Sim and Out sets, versus 65.15% for the best text-only baseline MSCLM and 67.13% for the best multimodal baseline DivE.
- On a 500-sample Out-set subset, DCIGN (54.4%) outperforms GLM4V (52.8%), InternVL2-Pro (51.8%), Qwen1.5 (52.6%) and GPT-4o (53.8%).
- Removing both contrastive losses drops average accuracy to 34.07%, the largest ablation drop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GCN for candidate-idiom graph modeling (ablation) | DCIGN · MChIRC Avg | accuracy: 51.47 -> 73.00 | Helped |
| Add coarse contrastive learning loss (Lccl) (ablation) | DCIGN · MChIRC Avg | accuracy: 34.07 -> 73.00 | Helped |
| Add fine contrastive learning loss (Lfcl) (ablation) | DCIGN · MChIRC Avg | accuracy: 65.93 -> 73.00 | Helped |
| Add text self-attention (TSA) (ablation) | DCIGN · MChIRC Avg | accuracy: 63.81 -> 73.00 | Helped |
| Add vision self-attention (VSA) (ablation) | DCIGN · MChIRC Avg | accuracy: 64.09 -> 73.00 | Helped |
| Add coarse and fine contrastive learning losses together (ablation) | DCIGN · MChIRC Avg | accuracy: 34.16 -> 73.00 | Helped |
| Add text and vision self-attention together (ablation) | DCIGN · MChIRC Avg | accuracy: 34.46 -> 73.00 | Helped |
| Change the coarse/fine contrastive-loss weights (α, β) across the reported settings (ablation) | DCIGN · MChIRC Avg | accuracy: 71.27; 72.61; 70.55; 73.00; 70.79; 72.12; 70.78; 70.71; 72.86 | Mixed |
Surprising findings
- The Out set is substantially harder than other splits for all methods; the best model reaches only 60.25% there.
- Text-only PRIEM achieves the highest Sim-set accuracy among text baselines but the lowest Out-set accuracy among them.
Limitations
- Dataset is built from images crawled from Baidu and Sogou with manual annotation and covers only 2,926 idioms.
- MLLM comparison uses only 500 Out-set samples due to cost, so the comparison is limited.
Experimental setup
Datasets: MChIRC ChID
Benchmarks: MChIRC
Metrics: accuracy Avg-Acc
Baselines: BERT-WWM RoBERTa MacBERT PRIEM RISCF MSCLM VGG-16 CLIP DeiT MSCA Multi-view CLIP DivE
Builds on: BERT DeiT ChID MSCLM
Closest papers by meaning
- Mitigating Idiom Inconsistency: A Multi-Semantic Contrastive Learning Method for Chinese Idiom Reading Comprehension (AAAI 2024 · similarity 0.92)
- BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset (NeurIPS 2025 · similarity 0.85)
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.84)
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models (ICLR 2025 · similarity 0.84)
- MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks (ICLR 2026 · similarity 0.84)
- GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.