Research map / Detection and segmentation / Open-set and few-shot detection
M^3-VOS: Multi-Phase, Multi-Transition, and Multi-Scenery Video Object Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces M3-VOS, a video object segmentation benchmark of 471 videos focused on object phases and phase transitions, and ReVOS, a plug-and-play reverse-propagation refinement module built on Cutie.
Problem
Existing video object segmentation benchmarks and models largely handle single-phase objects and fail to track objects undergoing phase transitions (intra-phase appearance changes and cross-phase state changes such as melting or sublimation).
Approach
Defines object phases from visual and dynamic characteristics, builds a dense-annotated benchmark of phase transitions, and observes that disorder increases over time so reverse propagation from the final frame can reduce segmentation difficulty; ReVOS adds a booster, reverse memory, and readout fusion on top of a frozen Cutie backbone.
Key results
- On M3-VOS full, ReVOS (Cutie-based) reaches J 75.6, Jtr 66.5, Jcc 65.2, versus Cutie-base at 74.6/64.6/64.3.
- On the cross-phase subset, Cutie-ReVOS achieves the best Jtr/Jcc among evaluated models (CP Jtr 70.8 vs Cutie-base 69.7 in the table).
- Ablation: a booster raises J from 74.2 (without) to 75.7 (with) on the mid set; backward-only propagation gives Jtr 66.4 vs forward-only 64.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply ReVOS to Cutie-base | Cutie-base · M3-VOS full | J: 74.6 -> 75.6 | Helped |
| apply ReVOS to Cutie-base | Cutie-base · M3-VOS core | J: 64.6 -> 66.3 | Helped |
| apply ReVOS to Cutie-base | Cutie-base · VOST val | J: 40.8 -> 41.0 | Hurt |
| apply ReVOS to Cutie-base | Cutie-base · DAVIS'17 val | J: 85.6 -> 86.0 | Helped |
| apply ReVOS to Cutie-base | Cutie-base · YouTubeVOS-2019 val | G: 86.8 -> 86.8 | No clear effect |
| increase sliding window length from 30 to 60 (ablation) | ReVOS · M3-VOS mid | J: 75.7 -> 75.4 | Hurt |
| reduce sliding window length from 30 to 10 (ablation) | ReVOS · M3-VOS mid | J: 75.7 -> 75.6 | No clear effect |
| increase reverse interval from 30 to 60 (ablation) | ReVOS · M3-VOS mid | J: 75.7 -> 75.7 | No clear effect |
| reduce reverse interval from 30 to 10 (ablation) | ReVOS · M3-VOS mid | J: 75.7 -> 75.5 | No clear effect |
| add reverse propagation to forward-only Cutie (ablation) | Cutie-base · M3-VOS mid | J: 74.7 -> 75.4 | Helped |
| add readout fusion to backward propagation (ablation) | ReVOS · M3-VOS mid | J: 75.4 -> 75.7 | Helped |
| add booster (ablation) | ReVOS · M3-VOS mid | J: 74.2 -> 75.7 | Helped |
Surprising findings
- SAM2 achieves the best result on VOST but performs worse than Cutie-based methods on M3-VOS.
- Larger reverse interval improved performance while also increasing FPS in the ablation.
Experimental setup
Datasets: YouTubeVOS DAVIS BURST OVIS MOSE VOST DAVIS 2017 val YouTubeVOS-2019 val
Benchmarks: M3-VOS full M3-VOS core VOST val DAVIS 2017 val YouTubeVOS-2019 val
Metrics: J Jtr Jcc F G FPS
Baselines: DeAOT XMem RMem + DeAOT SAM2 Cutie-base
Builds on: Cutie XMem DeAOT SAM2 RITM
Closest papers by meaning
- Breaking the "Object" in Video Object Segmentation (CVPR 2023 · similarity 0.92)
- Using Diffusion Priors for Video Amodal Segmentation (CVPR 2025 · similarity 0.88)
- GMOS: Grounding Moving Object Segmentation in 3D Space and Time (NeurIPS 2026 · similarity 0.88)
- Multi-Granularity Video Object Segmentation (AAAI 2025 · similarity 0.87)
- Point-VOS: Pointing Up Video Object Segmentation (CVPR 2024 · similarity 0.87)
- Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.