Research map / Reinforcement learning / Visual and representation RL
Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RL
Fangwei Zhong, Kui Wu, Hai Ci, Chu-ran Wang, Hao Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text-conditioned segmentation masks from a VFM as the state instead of raw RGB (ablation) | Offline RL tracker · Complex Room | success rate: 0.03 -> 0.92 | Improved |
| add re-targeting mechanism (target shown in white) (ablation) | Ours (DEVA) · Complex Room | success rate: 0.57 -> 0.92 | Improved |
| add recurrent (LSTM) temporal encoder (ablation) | Ours (DEVA) · Complex Room | success rate: 0.60 -> 0.92 | Improved |
| use DEVA with text prompt rather than DINOv2 masks (ablation) | Offline RL tracker · Complex Room | success rate: 0.74 -> 0.92 | Improved |
| swap DEVA for SAM-Track as the VFM (ablation) | Ours · Complex Room | success rate: 0.92 -> 0.97 | Improved |
| use moderate action noise (level 2) in data collection rather than level 1 or 4 (ablation) | Ours · Complex Room (40k) | success rate: 0.18 (level 1), 0.82 (level 2), 0.45 (level 4) | Mixed |
| increase offline dataset from 20k to 80k steps (ablation) | Ours · Complex Room | success rate: 0.23 (20k), 0.82 (40k), 0.76 (80k) | Mixed |
| scale BC expert data from 200k to 1000k steps (ablation) | Behavior Cloning · Complex Room | success rate: 0.32 -> 0.49 | No clear effect |
| offline RL with multi-level noisy data instead of BC on expert data | Embodied visual tracker · Mean over 5 unseen environments | success rate: 0.32 -> 0.95 | Improved |
| train offline for 1 hour instead of online RL for 12-26 hours | Embodied visual tracker · Mean over 5 unseen environments | success rate: 0.95 vs 0.92 (RSPT), 0.86 (TS) | Mixed |
| add RNN for distractor robustness on unseen animal targets (ablation) | Ours · Animals (4D), Simple Room | success rate: 0.77 -> 0.84 | Improved |
Closest papers by meaning
- HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents (ICLR 2025 · similarity 0.87)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.87)
- VIKI‑R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Selective Visual Representations Improve Convergence and Generalization for Embodied AI (ICLR 2024 · similarity 0.86)
- Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics (NeurIPS 2025 · similarity 0.86)
- DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
