Research map / Vision-language and multimodal / Multimodal LLMs
Don't Overthink with Pixels: Efficient Reasoning for Segmentation
Song Wang, Gongfan Fang, Lingdong Kong, Xiangtai Li, Jianyun Xu, Sheng Yang, Qiang Li, Jianke Zhu, Xinchao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PixelThink, a reasoning segmentation approach that uses reinforcement fine-tuning to reduce overthinking by adapting reasoning length to task complexity.
Problem
Reasoning segmentation methods fine-tune MLLMs with image-text-mask data and reinforcement fine-tuning, but they overthink and produce uniformly verbose reasoning chains regardless of task complexity.
Approach
Abstract is truncated; the stated idea is a simple approach that makes reasoning length adaptive to task complexity during reinforcement fine-tuning for segmentation, to avoid uniformly verbose chains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add difficulty-aware token budget (alone) (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU / tokens: gIoU 59.65 -> 62.07; tokens 86.92 -> 39.95 | Improved |
| add uncertainty-guided budget (alone) (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU / tokens: gIoU 59.65 -> 59.54 (cIoU 50.92 -> 52.37); tokens 86.92 -> 42.41 | Mixed |
| combine difficulty and uncertainty (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU (%): 59.65 -> 63.81 | Improved |
| use 3-level difficulty split instead of 2 or 4 (ablation) | PixelThink · ReasonSeg-DIFF val | tokens / gIoU: 2-level: 61.78 tok, 64.38 gIoU; 3-level: 46.98 tok, 63.81; 4-level: 76.89 tok, 62.28 | Mixed |
| constrain length of medium-difficulty samples (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU (%): 63.81 -> 60.92 | Hurt |
| raise uncertainty weight alpha from 25 to 35 (ablation) | PixelThink · ReasonSeg-DIFF val | tokens / gIoU: tokens 46.98 -> 71.10; gIoU 63.81 -> 62.66 | Hurt |
| replace top-2 margin with entropy as uncertainty (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU (%): 63.81 -> 62.43 | Hurt |
| remove reasoning entirely (no-thinking) (ablation) | Seg-Zero · ReasonSeg-DIFF val | gIoU (%): 63.81 -> 60.19 (RL) / 60.37 (prompt) | Hurt |
| add prompt-level 64-token limit | Seg-Zero · ReasonSeg-DIFF val | gIoU (%): 61.63 -> 62.50; tokens 90.79 -> 57.62 | Improved |
| add L1-Exact rigid length reward | Seg-Zero · ReasonSeg-DIFF val | gIoU (%): 61.63 -> 40.97 | Hurt |
| add soft length penalty with difficulty and uncertainty | Seg-Zero · Inference latency | latency (ms): 1282.48 -> 892.93 | Improved |
| label difficulty with direct MLLM prompting rather than three-dimension decomposition (ablation) | PixelThink · ReasonSeg-DIFF val | gIoU (%): 63.81 -> 56.32 | Hurt |
Closest papers by meaning
- DR$^2$Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models (ICML 2026 · similarity 0.91)
- LISA: Reasoning Segmentation via Large Language Model (CVPR 2024 · similarity 0.90)
- SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation (ICLR 2026 · similarity 0.90)
- LENS: Learning to Segment Anything with Unified Reinforced Reasoning (AAAI 2026 · similarity 0.90)
- To Think or Not to Think: Pre-Decisional Reasoning Budgets for Referring Audio-Visual Segmentation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
