Research map / Vision-language and multimodal / Multimodal LLMs

Fast Reasoning Segmentation for Images and Videos

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Distills a digital-twin-based reasoning segmentation pipeline from an 8B Qwen3 teacher into 1.7B and 0.6B Qwen3 students via SFT on teacher reasoning chains followed by GRPO with format, IoU, and LLM-judged reasoning-consistency rewards.

Problem

Reasoning segmentation requires multimodal LLMs with billions of parameters, too heavy for edge devices, and standard distillation fails to preserve multi-step reasoning chains.

Approach

Decouple perception from reasoning by having the LLM reason over a pre-computed digital twin (SAM-2 masks, Depth-Anything-2 depth stats, OWLv2 labels serialized as JSON) rather than visual tokens, then distill the teacher's reasoning chains into small Qwen3 students using SFT followed by RL with a reward that scores agreement with the teacher's reasoning.

Key results

• FastReasonSeg-8B teacher reaches 0.809 avg J on JiTBench Level 1, versus 0.792 for JiT with GPT-4o.
• FastReasonSeg-0.6B-Distill reaches 7.79 FPS with 2.1 GB memory and 0.6B LLM parameters, per the efficiency table.
• On ReasonSeg short queries, FastReasonSeg-8B reaches 0.746 gIoU and 0.713 cIoU, reported as gains of 0.128 and 0.129 over JiT with GPT-4o.

Surprising findings

• Replacing digital twin representations with direct visual tokens drops average J on JiTBench from 0.760 to 0.588 for the 1.7B distilled model.
• Direct RL without SFT (0.669 avg J) performs worse than SFT alone (0.716), so the SFT stage is needed before RL.

Limitations

• Relies on pre-computed digital twin representations, so real-time construction is not yet implemented.
• Reasoning reward depends on GPT-4o as an external LLM judge.
• Baselines on video are adapted from image methods via frame-by-frame inference.

Experimental setup

Datasets: RefCOCOg ReasonSeg ReVOS JiTBench RVTBench LLM-Seg40K

Benchmarks: JiTBench RVTBench ReasonSeg LLM-Seg40K

Metrics: J (region similarity) F (contour accuracy) gIoU cIoU FPS latency memory FLOPs

Baselines: LISA VISA LLM-Seg SegZero CoReS JiT

Builds on: GRPO just-in-time digital twin (JiT) SegZero LISA LoRA

Closest papers by meaning

• LISA: Reasoning Segmentation via Large Language Model (CVPR 2024 · similarity 0.89)
• DR$^2$Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models (ICML 2026 · similarity 0.88)
• SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation (CVPR 2026 · similarity 0.88)
• Discriminative Perception via Anchored Description for Reasoning Segmentation (CVPR 2026 · similarity 0.87)
• CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation (ICLR 2026 · similarity 0.87)
• The Devil is in Temporal Token: High Quality Video Reasoning Segmentation (CVPR 2025 · similarity 0.87)