Research map / Reinforcement learning / MDPs and planning
Tracking as Online Decision-Making: Learning a Policy From Streaming Videos With Reinforcement Learning
James Supancic,III, Deva Ramanan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace online heuristics with a learned policy (offline heuristics plus Q-learning) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .09 -> .30 | Improved |
| replace online heuristics with offline-heuristic classifier using track heuristic only (ablation) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .09 -> .13 | Improved |
| use offline update heuristic only (ablation) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .09 -> .14 | Improved |
| use both offline track and update heuristics (ablation) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .09 -> .20 | Improved |
| add Q-learning refinement on top of offline heuristics (ablation) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .20 -> .30 | Improved |
| learn only the appearance update action with Q-learning (ablation) | p-track (FCNT) · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .28 | Improved |
| use CCOT TRACK/UPDATE modules instead of FCNT (ablation) | p-track · Long-term Internet videos (16 held-out, >=5,000 frames) | F1: .30 -> .36 | Improved |
| initialize Q-functions from offline heuristics with supervised regularizer (ablation) | deep Q-learning tracker · Long-term Internet videos (16 held-out, >=5,000 frames) | convergence | Improved |
| train a dedicated short-term policy on dissimilar short-term videos (p-track-short) instead of long-term policy (ablation) | p-track · OTB-2013 / VOT-2016 | accuracy | Improved |
| train policy on long-term Internet videos and test on short-term benchmarks (ablation) | p-track-long · OTB-2013 / VOT-2016 | accuracy | Mixed |
| train on more videos beyond 8,000-12,000 (ablation) | Q-learning tracker · Long-term Internet videos (16 held-out, >=5,000 frames) | F1 | No clear effect |
| replace bounding-box labels with binary correct/incorrect feedback | annotation · annotation throughput | frames per minute: 34 -> 1200 | Improved |
Closest papers by meaning
- Visual Tracking via Hierarchical Deep Reinforcement Learning (AAAI 2021 · similarity 0.89)
- Online Decision Based Visual Tracking via Reinforcement Learning (NeurIPS 2020 · similarity 0.88)
- Real-time 'Actor-Critic' Tracking (ECCV 2018 · similarity 0.88)
- Action-Decision Networks for Visual Tracking With Deep Reinforcement Learning (CVPR 2017 · similarity 0.88)
- A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- Partially Observable Reinforcement Learning with Memory Traces (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
