Research map / Vision-language and multimodal / Multimodal LLMs

Uni-CoT: Towards Unified Chain-of-Thought Reasoning Across Text and Vision

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Uni-CoT fine-tunes the unified BAGEL model with a macro-micro hierarchical multimodal chain-of-thought (planning/summarization plus MDP-style self-reflection with masked attention) on ~31K CoT samples, reporting gains on GenEval, WISE, KRIS and Jigsaw-R1.

Problem

Multimodal chain-of-thought requires generating both text and visual intermediate states, which inflates token counts (~9,000+ visual tokens per step) and makes training unstable and inference quadratic in trajectory length.

Approach

A two-level CoT: a macro planner and summarizer over interleaved text-image content, with micro-level subtask execution formulated as a Markov decision process (self-reflection conditioned only on the last state and subgoal, enforced via macro/micro attention masks), plus auxiliary objectives for action, next-state and reward prediction.

Key results

• On GenEval, Uni-CoT reaches 0.83 overall vs. 0.79 for BAGEL.
• On WISE, Uni-CoT scores 0.75 overall, above Bagel-Think (0.70) and MetaQuery (0.55), though below GPT-4o (0.80).
• On KRIS, Uni-CoT reaches 68.00 overall, above GPT-4o (80.09 is higher; Uni-CoT surpasses Gemini-2.0 at 62.41) and BAGEL-Think (60.18).

Surprising findings

• Uni-CoT w/o CoT already improves over BAGEL on GenEval (0.81 vs 0.79), suggesting gains are not solely from the reasoning procedure.
• On RISE, Uni-CoT (12.5 overall) is only comparable to Gemini-2.0 (13.3) and well below GPT-4o (28.9).

Limitations

• Training data is small and predominantly synthetic, limiting generalization to real-world scenarios.
• Data generation relies on GPT-4o and other closed models for evaluation and editing.
• Performance on RISE and some understanding benchmarks (MMMU) is comparable to or slightly below baselines.

Experimental setup

Datasets: GeoPose3K PIPE ShareGPT-4o-image Echo-4o LLaVA-OneVision

Benchmarks: GenEval WISE GEdit-Bench KRIS RISE MME MMMU MMBench MathVista Jigsaw-R1

Metrics: overall score accuracy SC/PQ/O scores tokens per step

Baselines: BAGEL BAGEL-Think MetaQuery Janus-Pro-7B GPT-4o Gemini 2.0 Qwen2.5-VL-7B GPT-4.1-mini

Builds on: BAGEL Chain-of-Thought Rectified Flow Mixture-of-Transformer-Experts

Closest papers by meaning

• TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding (ICML 2026 · similarity 0.93)
• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.92)
• FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-and-Language Navigation (CVPR 2026 · similarity 0.91)
• Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning (NeurIPS 2025 · similarity 0.91)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
• Chain-of-Thought Guided Multi-Modal Object Re-Identification (CVPR 2026 · similarity 0.91)