Research map / Large language models / Agents, code and math
Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoMCTS, a Monte Carlo tree search that uses multiple MLLMs collectively to search reasoning paths, builds the Mulberry-260k dataset, and trains Mulberry o1-like reasoning/reflection MLLMs via SFT.
Problem
MLLMs typically answer with brief direct predictions and fail on complex multi-step reasoning; standard tree search self-bootstrapped from a single MLLM gets trapped in low-quality homogeneous nodes and is inefficient for multimodal reasoning.
Approach
Collective MCTS where a group of MLLMs jointly expands candidate reasoning continuations, jointly scores them to position and prune erroneous steps, and backpropagates UCB-based statistics; negative sibling nodes are then used to build reflective reasoning trajectories for SFT.
Key results
- Mulberry-7B averages 58.9 across 8 benchmarks vs 54.7 for Qwen2-VL-7B (+4.2).
- CoMCTS reaches 80.2% search success rate with 12.7 average iterations, vs 63.8% for MCTS (42.1 iterations) and 66.2% for Omega-MCTS (24.3).
- Mulberry-LLaVA-8B improves LLaVA-NeXT-8B average from 39.7 to 50.7 (+11).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train Qwen2-VL-7B on Mulberry-260K | Qwen2-VL-7B (Mulberry-7B) · 8 benchmarks | AVG: +4.2% | Helped |
| train LLaMA-3.2-11B-Vision-Instruct on Mulberry-260K | LLaMA-3.2-11B-Vision-Instruct (Mulberry-11B) · 8 benchmarks | AVG: +7.5% | Helped |
| train Qwen2-VL-2B on Mulberry-260K | Qwen2-VL-2B (Mulberry-2B) · 8 benchmarks | AVG: +5.4% | Helped |
| train LLaVA-NeXT-8B on Mulberry-260K | LLaVA-NeXT-8B (Mulberry-LLaVA-8B) · 8 benchmarks | AVG: +11.0% | Helped |
| use CoMCTS instead of direct prediction | GPT-4o · Geo3K and GeoQA-Plus | Search Success Rate: 58.2 -> 80.2 | Helped |
| use CoMCTS rather than Omega-MCTS | GPT-4o tree search · Geo3K and GeoQA-Plus | Average Search Iteration: 24.3 -> 12.7 | Helped |
| use CoMCTS with GPT-4o only rather than direct prediction (ablation) | GPT-4o · Geo3K and GeoQA-Plus | Search Success Rate: 58.2 -> 63.8 | Helped |
| add Qwen2-VL-7B to the CoMCTS model group (ablation) | CoMCTS with GPT-4o · Geo3K and GeoQA-Plus | Search Success Rate: 63.8 -> 66.2 | Helped |
| add LLaMA-3.2-11B to the CoMCTS model group (ablation) | CoMCTS with GPT-4o and Qwen2-VL-7B · Geo3K and GeoQA-Plus | Search Success Rate: 66.2 -> 69.7 | Helped |
| add Qwen2-VL-72B as the fourth model in CoMCTS (ablation) | CoMCTS with three models · Geo3K and GeoQA-Plus | Search Success Rate: 69.7 -> 80.2 | Helped |
| add reflection data to training (ablation) | Mulberry · MathVista | score: 50.9 -> 51.7 | Helped |
Surprising findings
- Reflection data adds only 0.8 points on MathVista (50.9 to 51.7).
- Tree search with GPT-4o alone already lifts search success from 58.2% to 63.8%, and adding the small Qwen2-VL-7B still helps.
Limitations
- Ablation of reflection data shows only a small gain (0.8 points on MathVista).
- Search relies on GPT-4o and other large models as collective members, raising data construction cost.
- Ablations are run on a small 1K-sample subset of Geo3K and GeoQA-Plus.
Experimental setup
Datasets: MathV360K GEOS Geo3K GeoQA Plus DocVQA ChartQA ScienceQA TextVQA A-OKVQA VQA2.0
Benchmarks: MathVista MMStar MMMU ChartQA DynaMath HallBench MM-Math MME
Metrics: accuracy search success rate average search iteration MME sum score
Baselines: GPT-4o Claude-3.5 Sonnet LLaVA-Reasoner-8B Insight-V-8B LLaVA-CoT-11B MCTS ReST-MCTS Omega-MCTS Qwen2-VL-2B Qwen2-VL-7B Llama-3.2-11B-Vision-Instruct LLaVA-NeXT-8B
Builds on: Monte Carlo tree search ReST-MCTS Omega-MCTS STaR LLaVA-CoT
Closest papers by meaning
- Think 360deg: Beyond Depth: Evaluating the Width-centric Reasoning Capability of MLLMs (CVPR 2026 · similarity 0.86)
- MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization (ICLR 2026 · similarity 0.86)
- TSLM: Tree-Structured Language Modeling for Divergent Thinking (ICLR 2026 · similarity 0.85)
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.85)
- Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search (NeurIPS 2025 · similarity 0.85)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.