Research map / Large language models / Reasoning and chain of thought
CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CLIFT, which certifies URL-conditioned verification questions with a conformal tracker to densify GRPO rewards for web agents and to drive judge-free trajectory selection (CTS) at test time.
Problem
Web-agent RL suffers from sparse binary success rewards and costly frontier-LLM judges that are unavailable at deployment; test-time selection also lacks a judge-free signal.
Approach
A bank of polarity-tagged, URL-scoped natural-language verification questions is certified via Mondrian adaptive conformal tracking against a training-time judge; certified signed trust weights add a non-negative verifier term to per-step rewards and drive a conservative pairwise majority-vote selector over greedy and retry rollouts.
Key results
- On WAI 9-app slice, Gemma-4 + CLIFT + CTS (K=3) reaches 74.6% overall vs 61.8% base Gemma-4 and 70.1% for Gemini 3 Flash + BU.
- On VWA, GPT-5.5 + CLIFT + CTS (K=4) reaches 53.7 weighted average, narrowly above WALT's 52.9.
- On OM2W with canonical judge, GPT-5.5 high goes from 49.7 to 61.0 with CTS at K=4 (+11.3pp); Gemma-4 goes from 40.0 to 45.0.
Surprising findings
- Vanilla GRPO raises steps on successful trajectories (15.4 to 18.5) yet gains only 2.9 points, while CLIFT gains 10.8 points with fewer steps.
- The transferred OM2W bank certified 100 of 132 questions, including 25 negative-lift failure-mode questions.
Limitations
- Relies on a frontier judge (Claude Opus 4.6) during training and bank construction; judge bias cannot be detected beyond observed data.
- Transfer does not establish a universal bank across all web domains.
- VWA result exceeds WALT by only 0.8 points, within noise-level margin.
Experimental setup
Datasets: WebArena Infinity (WAI) training tasks synthesized VisualWebArena (VWA) training tasks synthesized Online Mind2Web (OM2W) rollouts for bank recalibration
Benchmarks: WebArena Infinity VisualWebArena Online Mind2Web
Metrics: success rate weighted average success rate
Baselines: Vanilla GRPO SGV WALT Claude Opus 4.6 Gemini 3 Flash + BU Qwen 3.5+ Kimi K2.5 GPT-5.5 low/high base
Builds on: GRPO Mondrian adaptive conformal inference CEGAR WebJudge
Closest papers by meaning
- How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.87)
- Safe and Scalable Web Agent Learning via Recreated Websites (ICML 2026 · similarity 0.87)
- AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines (ICML 2026 · similarity 0.87)
- WebGym: Scaling Training Environments for Long-Horizon Visual Web Agents with Realistic Tasks (CVPR 2026 · similarity 0.86)
- ReVeal: Self-Evolving Code Agents via Reliable Self-Verification (ICLR 2026 · similarity 0.86)
- REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.