Research map / Large language models / Reasoning and chain of thought

ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ARES, a two-stage multimodal adaptive-reasoning training pipeline (difficulty-aware cold-start SFT plus AEPO, an RL method using high window-entropy tokens as exploration triggers and a difficulty-aware entropy reward and dynamic KL) on Qwen2.5-VL-3B/7B.

Problem

Multimodal large reasoning models overthink easy problems and under-explore hard ones, producing either unnecessarily long traces or missed solutions.

Approach

Uses sliding-window token entropy (high window-entropy tokens) to decide when to explore, and a difficulty-bucketed hierarchical entropy reward with token-adaptive KL weights to decide how much to explore; cold-start data is selected so response length tracks pass rate.

Key results

• ARES-7B averages 55.9 across 10 multimodal benchmarks, versus 46.2 for the best open 7B baseline (Vision-G1).
• ARES-7B scores 61.7 on AIME25 and 65.0 on AIME24, where most 7B baselines score below 3.3 on AIME25.
• On the ablation table, the entropy reward alone yields 55.2 average and the full ARES-7B reaches 55.7, versus 53.4 for cold start plus GRPO.

Surprising findings

• Response length grows on the hardest benchmarks after RL (AIME25 rises from 16361 to 22618 tokens) while shrinking on easy ones such as GSM8K.

Limitations

• Relies on GPT-based semantic filtering to build the reasoning-trigger token vocabulary.
• Hyperparameters such as window size and the 95th-percentile threshold are chosen empirically.

Experimental setup

Datasets: FigureQA Super-CLEVR MAVIS TabMWP K12-Vista MMK12 GeoQA UniGeo Geometry3K MultiMath IconQA Grammer

Benchmarks: MathVerse-V MathVision MathVista DynaMath-W WeMath LogicVista MMMU MMMU-Pro CharXIV MMStar AIME24 AIME25

Metrics: pass@1 accuracy average@16 accuracy (AIME) average response length

Baselines: GPT-4.1 Gemini-2.5-Pro-Thinking Claude-4-Sonnet Doubao-1.5-Thinking-Vision-Pro Qwen2.5-VL-3B-Instruct FAST-3B VLAA-Thinker-3B Qwen2.5-VL-7B-Instruct OpenVLThinker-1.2-7B MM-Eureka-Qwen-7B GRPO DAPO

Builds on: GRPO DAPO Revisual-R1 DeepSeek-R1

Closest papers by meaning

• DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference (ICLR 2026 · similarity 0.91)
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning Models (ICML 2026 · similarity 0.89)
• Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.89)
• PEAR: Phase Entropy Aware Reward for Efficient Reasoning (ICLR 2026 · similarity 0.89)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.89)