Research map / Large language models / Language, safety and interpretability
Toolformer: Language Models Can Teach Themselves to Use Tools
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| self-supervised finetuning on API-call-augmented data with tools enabled | GPT-J · LAMA T-REx | accuracy: 34.9 -> 53.5 | Helped |
| enable calculator API calls (ablation) | Toolformer · ASDiv | accuracy: 14.8 -> 40.4 | Helped |
| finetune on API-augmented data (calls disabled at inference) (ablation) | GPT-J · ASDiv | accuracy: 9.6 -> 14.8 | Helped |
| enable Wikipedia search (ablation) | Toolformer · WebQS | accuracy: 18.9 -> 26.3 | Helped |
| Toolformer (6.7B) vs GPT-3 175B | GPT-J · TriviaQA | accuracy: 65.9 -> 48.8 | Hurt |
| finetune on CCNet with MT tool | GPT-J · MLQA | accuracy: 16.5 -> 13.5 (De); 15.2 -> 20.6 (Es) | Mixed |
| add calendar tool (ablation) | GPT-J · Dateset | accuracy: 5.9 -> 27.3 | Helped |
| finetune on API-augmented data instead of plain CCNet (ablation) | GPT-J · WikiText | perplexity: 10.3 -> 10.3 | No clear effect |
| increase API-call decoding k from 1 to 10 (ablation) | Toolformer · WebQS | accuracy: 19.3 -> 26.3 | Helped |
| scale model size | Toolformer with GPT-2 family · LAMA/math/QA average | accuracy | Mixed |
Closest papers by meaning
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (ICLR 2024 · similarity 0.89)
- Large Language Models as Tool Makers (ICLR 2024 · similarity 0.89)
- ToolACE: Winning the Points of LLM Function Calling (ICLR 2025 · similarity 0.88)
- Gorilla: Large Language Model Connected with Massive APIs (NeurIPS 2024 · similarity 0.88)
- CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets (ICLR 2024 · similarity 0.87)
- ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.