Research map / Large language models / Agents, code and math
From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning
Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee, Woohyung Lim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Teacher-Student adaptive difficulty scaling to base problems | ATAD (GPT-4o generator) · GPT-4o evaluated | accuracy (%): 94.29 -> 72.43 | Improved |
| add Teacher-Student adaptive difficulty scaling | ATAD (Gemini-2.0-Flash generator) · GPT-4o evaluated | accuracy (%): 83.29 -> 44.71 | Improved |
| add Orchestrator validation (ablation) | ATAD (GPT-4o generator) · GPT-4o problem approval rate | approval rate (%): 38.14 -> 87.14 | Improved |
| add Orchestrator validation (ablation) | ATAD (GPT-4o generator) · Coherence (1-5), GPT-4o reviewer | coherence: 3.71 -> 4.74 | Improved |
| add Orchestrator validation (ablation) | ATAD (Gemini-2.0-Flash generator) · Problem quality | validity/coherence/fairness: 5.00 -> 5.00 validity; 4.97 -> 5.00 coherence | No clear effect |
| add Orchestrator validation (ablation) | ATAD (GPT-4o generator) · accuracy of evaluators | accuracy (%): 68.29 -> 72.43 (GPT-4o) | Improved |
| increase number of generated samples from 350 to 7000 | ATAD (Gemini-2.0-Flash generator) · GPT-4o-mini accuracy | accuracy | No clear effect |
| evaluate with newer reasoning model GPT-o3-mini/o4-mini instead of GPT-4o | ATAD (GPT-4o generator) · Final problems | accuracy (%): GPT-4o 72.43 vs o3-mini 72.14, o4-mini 72.43 | No clear effect |
Closest papers by meaning
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.90)
- \$OneMillion-Bench: How Far are Language Agents from Human Experts? (NeurIPS 2026 · similarity 0.89)
- Fleet of Agents: Coordinated Problem Solving with Large Language Models (ICML 2025 · similarity 0.89)
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.89)
- Unbiased Evaluation of Large Language Models from a Causal Perspective (ICML 2025 · similarity 0.89)
- PACE: A Proxy for Agentic Capability Evaluation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
