Research map / Large language models / Agents, code and math
Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls
Zeyu Zhang, Guohao Li, Zhenchang Xing, Alexandros Apostolopoulos, Yu Lin Lee, Liang Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GATS (simulated refinement in Gecko) to the planning LLM | GPT-4o · BFCLv3 overall | overall accuracy (%): 76.93 -> 84.62 | Improved |
| add GATS to the planning LLM | GPT-5 · BFCLv3 overall | overall accuracy (%): 61.94 -> 66.08 | Improved |
| add GATS to the planning LLM | Gemini-3.0-pro · BFCLv3 overall | overall accuracy (%): 79.97 -> 85.19 | Improved |
| add GATS to the planning LLM | GPT-4o · tau2-bench overall | average pass@1 (%): 54.2 -> 60.7 | Improved |
| add GATS to the planning LLM | GPT-5.5 · tau2-bench overall | average pass@1 (%): 85.5 -> 87.9 | Improved |
| add GATS to the planning LLM | Deepseek-V3 · BFCLv3 overall | overall accuracy (%): 70.40 -> 72.90 | Improved |
| add GATS to GPT-4o for multi-turn base | GPT-4o · BFCLv3 multi-turn base | accuracy (%): 61.0 -> 72.0 | Improved |
| add GATS rather than Reflexion, Self-Refine, Best-of-N or Majority Voting | GPT-5-mini · tau2-airline | pass@1 (%): 60.0 -> 65.0 | Improved |
| collapse Gecko into a single prompt (Merge-to-one) (ablation) | GPT-4o · BFCLv3 multi-turn base | accuracy (%): 61.0 -> 58.0 | Hurt |
| add task feedback judge LLM (ablation) | Gecko · BFCLv3 multi-turn base | accuracy (%): 61.5 -> 72.0 | Improved |
| add task state estimation (ablation) | Gecko · BFCLv3 multi-turn base | accuracy (%): 68.0 -> 72.0 | Improved |
| add argument validation (ablation) | Gecko · BFCLv3 multi-turn base | accuracy (%): 70.5 -> 72.0 | Improved |
Closest papers by meaning
- AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.89)
- ToolACE: Winning the Points of LLM Function Calling (ICLR 2025 · similarity 0.89)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.89)
- AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition (ICML 2026 · similarity 0.89)
- Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments (ICLR 2026 · similarity 0.89)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
