Research map / Large language models / Agents, code and math
MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants
Zuhao zhang, Chengyue Yu, Yuante Li, Chenyi Zhuang, Linjian Mo, Shuai Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add source code as judge input (ablation) | MiniAppEval · Agreement with human labels, 200-item subset | accuracy (%): 70.66 -> 89.62 | Improved |
| add Playwright agent interaction (ablation) | MiniAppEval · Agreement with human labels, 200-item subset | accuracy (%): 66.48 -> 89.62 | Improved |
| add evaluation reference (ablation) | MiniAppEval · Agreement with human labels, 200-item subset | accuracy (%): 60.12 -> 89.62 | Improved |
| add evaluation reference (ablation) | MiniAppEval · Agreement with human labels, 200-item subset | precision (%): 89.47 -> 83.87 | Hurt |
| add evaluation reference (ablation) | MiniAppEval · Agreement with human labels, 200-item subset | F1 (%): 61.08 -> 84.55 | Improved |
| add double-blind judging (judge sees output before query) | MiniAppEval (Gemini-3-Pro-Preview) · Graphical queries (55), Gemini-3-Pro-Preview judge | accuracy (%): 81.82 -> 85.45 | Improved |
| add double-blind judging | MiniAppEval (GPT-5.2) · Graphical queries (55), GPT-5.2 judge | accuracy (%): 80.00 -> 83.63 | Improved |
| add double-blind judging | MiniAppEval (Claude-Opus-4.5) · Graphical queries (55), Claude-Opus-4.5 judge | accuracy (%): 78.18 -> 83.63 | Improved |
| scale Qwen3 from 32B to 235B-A22B | Qwen3 · MiniAppBench | pass rate (%): 0.66 -> 2.88 | Improved |
| scale GLM from 4.5-Air to 4.7 | GLM · MiniAppBench | pass rate (%): 7.09 -> 18.31 | Improved |
| use closed-source instead of open-source models | LLMs · MiniAppBench | avg pass rate (%): 6.19 -> 45.46 | Improved |
| raise task difficulty from Easy to Hard | all models average · MiniAppBench | pass rate (%): 34.05 -> 4.34 | Hurt |
Closest papers by meaning
- ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents (ICLR 2025 · similarity 0.87)
- Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale (ICML 2025 · similarity 0.87)
- TaskBench: Benchmarking Large Language Models for Task Automation (NeurIPS 2024 · similarity 0.87)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.87)
- AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use (ICML 2026 · similarity 0.87)
- AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
