Research map / Vision-language and multimodal / Multimodal LLMs

Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Rationale-Enhanced Decoding (RED), a training-free inference-time method that multiplies image-conditional and rationale-conditional next-token distributions to make LVLM chain-of-thought predictions better grounded on rationales.

Problem

LVLMs under multi-modal chain-of-thought prompting often ignore the content of generated rationales, with image tokens dominating and CoT sometimes degrading performance relative to direct answering.

Approach

Reformulates multi-modal CoT as KL-constrained reward maximization of rationale-conditional log-likelihood with the image-conditional distribution as reference; the closed-form optimum is a power-of-experts product p(y|x,q) x p(y|r,q)^lambda, implemented as a weighted sum of log-softmax logits with batched dual forward passes.

Key results

• CCoT + RED with Gemma-3-12B reaches 47.50 on GQA vs. 45.34 baseline and 44.50 CCoT (+2.16 over baseline).
• CoT + RED with Qwen2.5-VL-7B improves MathVista from 64.70 baseline to 70.60 (+5.90) and SEED-I text understanding by +38.83 over CoT's counterpart.
• On GQA with Qwen2.5-VL-7B, CCoT + RED with GPT-4 rationales reaches 63.11 while random rationales drop accuracy to 41.74 (-19.14 vs. CCoT+RED self), indicating rationale dependence.

Surprising findings

• Standard CoT and CCoT often degrade performance below direct answering on several LVLM-benchmark pairs (e.g., Qwen2.5-VL-7B on GQA drops ~14 points with CoT).
• Replacing rationales with irrelevant ones often barely changes CoT/CCoT accuracy, indicating LVLMs largely ignore rationale content.

Limitations

• Inference overhead from dual forward passes and increased GPU memory consumption.
• Best lambda depends on rationale type and must be tuned on validation data; very large lambda reverts toward rationale-only behavior.

Experimental setup

Datasets: GQA TextVQA MME SEED-I LLaVA-Bench MM-Vet MathVista MMMU MMMU-Pro MMHal POPE

Benchmarks: GQA LLaVA-Bench MME MMVet SEED-I TextVQA MathVista MMMU MMMU-Pro MMHal POPE

Metrics: accuracy MME perception/cognition score MMHal average score hallucination ratio inference latency

Baselines: Baseline direct decoding CoT CCoT VCD VCD + ICD

Builds on: chain-of-thought prompting KL-constrained reward maximization visual contrastive decoding CCoT contrastive decoding

Closest papers by meaning

• Latent Chain-of-Thought for Visual Reasoning (NeurIPS 2025 · similarity 0.92)
• Interleaved-Modal Chain-of-Thought (CVPR 2025 · similarity 0.91)
• Self-Prophetic Decoding to Unlock Visual Search in LVLMs (ICML 2026 · similarity 0.90)
• HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning (NeurIPS 2026 · similarity 0.90)
• TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding (ICML 2026 · similarity 0.90)
• Multi-Modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models (AAAI 2024 · similarity 0.90)