Research map / Reinforcement learning / Visual and representation RL
Focus-Then-Decide: Segmentation-Assisted Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Focus-Then-Decide (FTD), which segments observations with MobileSAM, uses an attention selector to weight task-relevant segments, and trains SAC on the selected frame with reward and inverse-dynamics self-supervised losses, outperforming baselines on noisy DMC and Franka-reach.
Problem
Visual RL agents learn inefficiently in noisy, distraction-filled environments because they must process all pixels, including task-irrelevant ones.
Approach
Treat observations as collections of objects obtained from a foundational segmentation model (MobileSAM), use a multi-head attention selector over segments to recombine only task-relevant segments into a focused frame, and train it end-to-end with SAC plus reward-prediction and inverse-dynamics losses.
Key results
- FTD ranks first on five of eight DMC distracted tasks, e.g. walker-walk 395.7 ± 48.9 vs 145.9 ± 24.9 for the next best baseline shown in Table 1.
- On Franka-reach, all six baselines gain almost no reward while FTD learns to catch the ball.
- The attention selector's success rate exceeds 90% in seven of eight tasks; SAM-selector success correlates with FTD performance (Pearson 0.72).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FTD instead of Q2-learning | visual RL with distracted observations · pendulum-swingup | return: 546.4 ± 366.8 -> 498.8 ± 282.8 | Hurt |
| use FTD instead of MICo | visual RL with distracted observations · cartpole-swingup | return: 289.0 ± 47.5 -> 207.3 ± 26.1 | Hurt |
| use FTD instead of MICo | visual RL with distracted observations · finger-spin | return: 494.9 ± 219.9 -> 591.5± 146.0 | Helped |
| use FTD instead of Q2-learning | visual RL with distracted observations · hopper-stand | return: 249.7 ± 51.5 -> 112.0 ± 41.6 | Hurt |
| use FTD instead of Q2-learning | visual RL with distracted observations · hopper-hop | return: 43.9 ± 15.8 -> 65.0 ± 26.6 | Helped |
| use FTD instead of Q2-learning | visual RL with distracted observations · cheetah-run | return: 60.6 ± 29.0 -> 228.9± 43.1 | Helped |
| use FTD instead of Denoised (S) | visual RL with distracted observations · walker-walk | return: 145.9 ± 24.9 -> 395.7± 48.9 | Helped |
| use FTD instead of MICo | visual RL with distracted observations · walker-run | return: 81.8 ± 46.2 -> 185.8 ± 12.3 | Helped |
| add the attention selector (ablation) | FTD · finger-spin | return | Helped |
| add the self-supervised objectives (ablation) | FTD · finger-spin | return | Helped |
| add both the attention selector and self-supervised objectives (ablation) | FTD · finger-spin | return | Helped |
Surprising findings
- Simply feeding segmentation outputs stacked on channels to SAC (FTD w/o SEL & SSO) yields almost no reward, so the attention selector is essential.
- DrQ-v2 hardly improves during training under video distractions.
Limitations
- Performance depends on segmentation accuracy of the foundation model; pendulum-swingup, cartpole-swingup and hopper-stand have low overall success rates.
- Selector is not permutation invariant and must handle a variable number of segments.
- FTD underperforms MICo and Q2-learning on simple-shaped object tasks with sparse rewards.
Experimental setup
Datasets: DeepMind Control Suite Franka Emika Robotics (RL-ViGen) 80 training video clips for background distraction 20 test video clips
Benchmarks: DeepMind Control Suite (8 tasks with video distractors) Franka-reach
Metrics: episode return segmentation success rate
Baselines: DrQ-v2 DBC MICo Q2-learning Denoised-MDP (D) Denoised-MDP (S)
Builds on: Soft Actor-Critic Segment Anything Model Denoised MDP DBC inverse dynamics prediction
Closest papers by meaning
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.88)
- LENS: Learning to Segment Anything with Unified Reinforced Reasoning (AAAI 2026 · similarity 0.88)
- Train the Agent, Not the Expert: Learning to Harness Heterogeneous Experts for Multi-Turn Visual Reasoning (NeurIPS 2026 · similarity 0.87)
- Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- Grounded Reinforcement Learning for Visual Reasoning (NeurIPS 2025 · similarity 0.87)
- SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.