Research map / Large language models / Agents, code and math
CUARewardBench: A Benchmark for Evaluating Reward Models for Computer-Using Agents
Haojia Lin, Xiaoyu Tan, Yulei Qin, Zihan Xu, Yuchen Shi, Zongyi Li, Gang Li, Shaofei Cai, Siqi Cai, Yuzheng Cai, Chaoyou Fu, Ke Li and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use strict unanimous voting instead of majority voting (ablation) | Q32-s + G106-s · CUARewardBench ORM | NPV: 68.5 -> 84.2 | Improved |
| use strict unanimous voting instead of majority voting (ablation) | Q32-s + G106-s · CUARewardBench ORM | specificity: 93.2 -> 72.2 | Hurt |
| add GLM-4.5V zerogui prompt member to the ensemble (ablation) | Q32-s + G106-s (majority) · CUARewardBench ORM | NPV: 68.5 -> 84.8 | Improved |
| add GLM-4.5V zerogui prompt member to the ensemble (ablation) | Q32-s + G106-s (strict unanimous) · CUARewardBench ORM | NPV: 84.2 -> 93.3 | Improved |
| add GLM-4.5V zerogui prompt member to the ensemble (ablation) | Q32-s + G106-s (strict unanimous) · CUARewardBench ORM | precision: 90.1 -> 89.8 | No clear effect |
| add GLM-4.5V opencua_reflector prompt member to the ensemble (ablation) | Q32-s + G106-s (strict unanimous) · CUARewardBench PRM | precision: 75.6 -> 81.7 | Improved |
| add GLM-4.5V opencua_reflector prompt member to the ensemble (ablation) | Q32-s + G106-s (strict unanimous) · CUARewardBench PRM | NPV: 69.1 -> 85.1 | Improved |
| use sewsm prompt instead of zerogui prompt (ablation) | Qwen2.5VL-32B · CUARewardBench ORM | precision: 70.9 -> 80.0 | Improved |
| use sewsm prompt instead of zerogui prompt (ablation) | Qwen2.5VL-32B · CUARewardBench ORM | NPV: 76.1 -> 71.7 | Hurt |
| use sewsm prompt instead of opencua_reflector prompt (ablation) | Qwen2.5VL-32B · CUARewardBench PRM | precision: 60.3 -> 65.3 | Improved |
| use sewsm prompt instead of opencua_reflector prompt (ablation) | Qwen2.5VL-32B · CUARewardBench PRM | NPV: 64.8 -> 62.4 | Hurt |
| CUA post-training (GUI-OWL-7B vs Qwen2.5VL-7B) | Qwen2.5VL-7B · CUARewardBench ORM (zerogui) | overall accuracy: 62.1 -> 68.4 | Improved |
Closest papers by meaning
- OpenCUA: Open Foundations for Computer-Use Agents (NeurIPS 2025 · similarity 0.89)
- Computer Use at the Edge of the Statistical Precipice (NeurIPS 2026 · similarity 0.87)
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use (ICML 2026 · similarity 0.87)
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.86)
- Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness (ICLR 2026 · similarity 0.86)
- ProRe: A Proactive Reward System for GUI Agents via Reasoner–Actor Collaboration (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
