Research map / Large language models / Agents, code and math
Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Confucius, a tool-learning framework that trains LLaMA-7B with an easy-to-difficult three-stage curriculum (warm-up, in-category, cross-category) plus iterative self-instruct guided by perplexity-based introspective feedback, evaluated on seen/unseen toolsets.
Problem
Open-source LLMs trained for tool use mostly learn to execute human-provided minimal toolsets and fail to select correct tools from large retrieved candidate sets; self-instruct data ignores differences in tool complexity and generalizes poorly to unseen tools.
Approach
A multi-stage curriculum that gradually adds related and retrieved distractor tools to the candidate set, combined with an iterative data-generation loop that uses high-perplexity generated instances as seeds for further self-instruct generation, so training focuses on intricate tools the model misuses.
Key results
- On the seen toolset, Confucius (LLaMA-7B) reaches 88.61 tool selection vs 83.62 for ChatGPT (4.99 absolute improvement).
- On the unseen toolset, Confucius reaches 59.79 tool selection and 63.65 compositional reasoning, outperforming ChatGPT (57.65) and GPT4Tools (44.58 / 46.21).
- Ablations show removing the cross-category stage causes the largest tool-selection drop (88.61 to 83.49 on seen).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with Confucius instead of ChatGPT | LLaMA-7B · Seen Toolset | Tool Selection (NDCG): 83.62 -> 88.61 | Helped |
| train with Confucius instead of ChatGPT | LLaMA-7B · Unseen Toolset | Tool Selection (NDCG): 57.65 -> 59.79 | Helped |
| add warm-up training (ablation) | Confucius · Seen Toolset | Parameter Correctness: 70.16 -> 77.72 | Helped |
| add in-category training (ablation) | Confucius · Seen Toolset | Tool Selection (NDCG): 85.73 -> 88.61 | Helped |
| add cross-category training (ablation) | Confucius · Seen Toolset | Tool Selection (NDCG): 83.49 -> 88.61 | Helped |
| add cross-category training (ablation) | Confucius · Unseen Toolset | Parameter Correctness: 40.70 -> 50.21 | Helped |
| add ISIF instead of training without ISIF (ablation) | Confucius · Seen Toolset | Tool Selection (NDCG): 83.52 -> 88.61 | Helped |
| add ISIF instead of training without ISIF (ablation) | Confucius · Unseen Toolset | Parameter Correctness: 41.45 -> 50.21 | Helped |
| increase ISIF update percentage from 20% to 30% (ablation) | Confucius · Seen and Unseen Toolsets | Overall performance | Hurt |
Surprising findings
- Removing the cross-category training stage causes the largest drop in tool selection, underscoring the value of retriever-based distractor candidates.
- Increasing the update percentage beyond 20% degrades performance, which the authors attribute to overfitting a few specific tools.
Limitations
- Tool-use data is generated by ChatGPT and evaluated on a synthetic test set built from the same generation pipeline, so real-world generalization is not directly measured.
- Human evaluation covers only 50 sampled cases with three annotators.
- The initial dataset includes some noisy instances (about 10% flagged invalid in quality review).
Experimental setup
Datasets: Self-generated tool-use dataset (110 tools, 72,000 instances)
Benchmarks: Seen toolset test (2,000 instances) Unseen toolset test (2,000 instances)
Metrics: tool selection (NDCG) parameter correctness compositional reasoning (ROUGE-L over tool order) interaction fluency (ROUGE-1/2/L avg) human executability and fluency
Baselines: ChatGPT Claude Text-davinci-003 ChatGLM-6B ChatGLM2-6B Llama-7B Llama2-7B Vicuna-7B Vicuna-13B GPT4Tools ToolLLaMA Toolformer
Builds on: Self-Instruct Toolformer GPT4Tools ToolBench ToolLLaMA
Closest papers by meaning
- From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions (ICLR 2025 · similarity 0.87)
- Large Language Models as Tool Makers (ICLR 2024 · similarity 0.85)
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (ICLR 2024 · similarity 0.85)
- Teaching Large Language Models to Translate with Comparison (AAAI 2024 · similarity 0.85)
- CITI: Enhancing Tool Utilizing Ability in Large Language Models Without Sacrificing General Performance (AAAI 2025 · similarity 0.85)
- Cross-model Control: Improving Multiple Large Language Models in One-time Training (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.