Research map / Large language models / Agents, code and math
TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
Pengfei He, Zhenwei Dai, Bing He, Hui Liu, Xianfeng Tang, Hanqing Lu, Juanhui Li, Jiayuan Ding, Subhabrata Mukherjee, Suhang Wang, Yue Xing, Jiliang Tang and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hard indirect queries instead of simple explicit queries (ablation) | Claude-4 · TRAJECT-Bench parallel | trajectory exact match: 0.846 -> 0.445 | Hurt |
| sequential dependency instead of parallel independent tools (ablation) | Gemini-2.5-pro · TRAJECT-Bench simple queries | trajectory exact match: 0.851 -> 0.807 | Hurt |
| increase number of tools in the ground-truth trajectory (ablation) | all evaluated LLMs · TRAJECT-Bench | trajectory exact match | Hurt |
| add retrieval (embedding models) when the pool is already domain-restricted, simple queries (ablation) | Claude-3.7 and Claude-4 · TRAJECT-Bench parallel simple | trajectory exact match: Claude-4: 0.846 -> 0.852 (bge-large) | No clear effect |
| replace domain tools with retrieved tools on hard queries (ablation) | Claude-4 · TRAJECT-Bench parallel hard | trajectory exact match: 0.445 -> 0.031 (bge-large, domain pool) | Hurt |
| retrieve from all tools instead of domain pool (ablation) | Claude-4 with bge-large · TRAJECT-Bench parallel simple | trajectory exact match: 0.852 -> 0.785 | Hurt |
| use model-native agentic tool calling instead of tools in context (ablation) | Claude-4, Gemini-2.5-pro, DeepSeek, Kimi-k2 · TRAJECT-Bench parallel | trajectory exact match: Claude-4 simple 0.846 -> 0.832 | No clear effect |
| wrap Claude-4 with ReAct and domain tools | Claude-4 · TRAJECT-Bench parallel hard | trajectory exact match: 0.445 -> 0.463 | Improved |
| retrieve tools at every ReAct turn instead of once (ablation) | ReAct · TRAJECT-Bench parallel hard | trajectory exact match: Claude-4: 0.031 -> 0.472; Claude-3.7: 0.093 -> 0.296 | Improved |
| static one-time retrieval in ReAct instead of domain tools on hard queries (ablation) | ReAct · TRAJECT-Bench parallel hard | trajectory exact match: Claude-4: 0.463 -> 0.031 | Hurt |
Closest papers by meaning
- Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents (ICML 2026 · similarity 0.93)
- T1: A Tool-Oriented Conversational Dataset for Multi-Turn Agentic Planning (NeurIPS 2025 · similarity 0.90)
- TaskBench: Benchmarking Large Language Models for Task Automation (NeurIPS 2024 · similarity 0.90)
- Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents (AAAI 2026 · similarity 0.90)
- AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.89)
- LLM Agents Already Know When to Call Tools - Even Without Reasoning (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
