Research map / Video / Video-language understanding
Reward Modeling for Computer-Using Agent from Video Execution
Linxin Song, Jieyu Zhang, Huanxin Sheng, Taiwei Shi, Rahul Gupta, Yang Liu, Ranjay Krishna, Jian Kang, Jieyu Zhao
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on ExeVR-53k execution videos | Qwen3-VL 8B · ExeVR-Bench | accuracy: 67.7 -> 84.7 | Improved |
| use dense execution video instead of sparse screenshots (ablation) | Qwen3-VL · ExeVR-Bench Ubuntu (Human) | accuracy: AER 58.5 -> ExeVRM 4B 81.0 | Improved |
| reduce resolution to 360p to fit all key frames (ablation) | SE-WSM / ZeroGUI · ExeVR-Bench Ubuntu (Human) | accuracy: SE-WSM 48.5 vs AER 55.0 (8B) | Hurt |
| raise input resolution from 360p to 720p with STP+TTP (ablation) | ExeVRM 8B · ExeVR-Bench | accuracy: 81.5 -> 84.7 | Improved |
| raise input resolution from 360p to 720p with STP+TTP (ablation) | ExeVRM 4B · ExeVR-Bench | accuracy: 79.3 -> 80.1; recall 77.8 -> 82.5; precision 80.6 -> 79.2 | Mixed |
| apply temporal token pruning alone vs spatial alone (ablation) | ExeVRM 4B · ExeVR-Bench | accuracy: STP only 77.6 -> TTP only 80.3 | Improved |
| add STP on top of TTP (ablation) | ExeVRM 4B · ExeVR-Bench | recall: accuracy 80.3 -> 80.1; recall 79.3 -> 82.5; precision 81.3 -> 79.2 | Mixed |
| combine STP and TTP for efficiency (ablation) | ExeVRM 4B · 50 frames | peak GPU memory: TTP-only ~68 GiB -> STP+TTP ~49.5 GiB | Improved |
| scale ExeVRM from 4B to 8B (ablation) | ExeVRM · ExeVR-Bench | accuracy: 80.1 -> 84.7 | Improved |
| train reward model on execution video for temporal attribution | ExeVRM 8B · ExeVR-Bench | tIoU: Gemini 3 Pro 0.2494 -> 0.3332 | Improved |
Closest papers by meaning
- Watch and Learn: Learning to Use Computers from Online Videos (CVPR 2026 · similarity 0.87)
- ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding (NeurIPS 2025 · similarity 0.87)
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use (ICML 2026 · similarity 0.87)
- Boosting Virtual Agent Learning and Reasoning: A Step-Wise, Multi-Dimensional, and Generalist Reward Model with Benchmark (ICML 2025 · similarity 0.86)
- OpenCUA: Open Foundations for Computer-Use Agents (NeurIPS 2025 · similarity 0.86)
- IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
