Research map / Vision-language and multimodal / Multimodal LLMs

Teach-to-Reason: Competition-Guided Reasoning with a Self-Improving Teacher

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Teach-to-Reason (T2R), which iteratively self-improves a Teacher via pairwise self-competition and uses a case-wise, competition-guided GRPO reward to train a Reasoner for chest X-ray open-ended VQA CoT quality.

Problem

Answer-level RL rewards give coarse, sparse supervision for chain-of-thought quality in chest X-ray VQA, and GRPO advantages collapse to zero when group rewards are identical.

Approach

Replace fixed rubric or single-CoT judge scores with pairwise LLM-judge competition against a self-improving Teacher's reference CoTs; preserve the task-reward positive/negative partition when informative and reconstruct advantages from competition scores when the group reward is degenerate.

Key results

• At 2B, T2R-R3 reaches 31.73 on Med-CXR and 40.37 on Covid19 vs. Frozen Teacher 21.74 and 31.88; T2R achieves best on five of six datasets at 2B.
• At 4B, T2R-R2 reaches 46.96 on VQA-RAD and 49.43 on Covid19, best on five of six datasets.
• In SFT distillation at 2B, T2R-R3 rollouts give a student that beats Frozen Teacher on most benchmarks (e.g., Med-CXR 26.44 vs 23.71).

Surprising findings

• Frozen Teacher competition scores collapse to near zero when the fixed teacher is stronger (T2 or T3), so comparison supervision becomes sparse.
• Directly injecting competition scores into the reward underperforms the case-wise design.

Limitations

• Depends on judge reliability; biased comparisons could misdirect the Reasoner.
• Substantially higher inference cost than RLVR due to K teacher CoTs and pairwise judging per sample.
• Frozen-teacher comparisons show supervision becomes sparse when the teacher is much stronger than the reasoner.

Experimental setup

Datasets: RexVQA

Benchmarks: Medical-CXR-VQA VQA-RAD SLAKE MIMIC-CXR-VQA Covid19_heywhale Chest_X-Ray_PA

Metrics: accuracy

Baselines: RLVR Only Judge Reward Frozen Teacher Base Model

Builds on: GRPO RLVR Qwen3-VL

Closest papers by meaning

• T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering (AAAI 2024 · similarity 0.84)
• Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.84)
• Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search (NeurIPS 2025 · similarity 0.84)
• Self-Aligned Reward: Towards Effective and Efficient Reasoners (ICLR 2026 · similarity 0.84)
• The Signal is in the Steps: Local Scoring for Reasoning Data Selection (ICML 2026 · similarity 0.84)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning (NeurIPS 2025 · similarity 0.84)