← All papers

Toolformer: Language Models Can Teach Themselves to Use Tools

Schick, Dwivedi-Yu, Dessì et al. (Meta) · 2023 · NeurIPS 2023

PromptingRead on arXiv

Trains language models to autonomously decide when and how to call external tools (calculator, search, calendar, etc.) by self-generating tool-use annotations and filtering by usefulness.

Key Idea

Toolformer teaches an LLM to insert API calls into its own text when doing so would improve its predictions. Unlike ReAct (which uses few-shot prompting), Toolformer fine-tunes the model on self-generated tool-use examples, making tool use a native capability rather than a prompted behavior.

How It Works

  1. Sample tool calls: Given a text, the model generates candidate API calls (e.g., [Calculator(350/7)]) at various positions using few-shot prompting
  2. Execute: Each candidate call is executed to get the result (e.g., → 50)
  3. Filter: Keep only calls where inserting the API result reduces perplexity on subsequent tokens, i.e., the tool actually helped
  4. Fine-tune: Train the model on the filtered dataset, so it learns when tools are genuinely useful
  5. At inference, the model naturally generates [ToolName(args)] tokens when it would benefit from external information

Why It Matters

  • Self-supervised tool learning: no human annotation of when to use tools, the model figures it out from perplexity reduction
  • Demonstrated tool use across 5 tools: calculator, Q&A system, search engine, translation, and calendar
  • Showed that even a 6.7B model with tools can outperform much larger models (GPT-3 175B) on tasks requiring factual knowledge or computation
  • Conceptual foundation for function calling in GPT-4, Claude, and Gemini, commercial APIs formalized what Toolformer prototyped
  • Key insight: models know when they don't know, perplexity spikes signal when tool use would help

Key Takeaways for Interviews

  • Toolformer = self-supervised fine-tuning for tool use via perplexity-based filtering
  • Unlike ReAct (prompted, sequential), Toolformer bakes tool use into the model weights, no special prompting needed at inference
  • The perplexity filter is the key innovation: only keep tool calls that actually improve next-token prediction
  • Limitation: fixed set of tools defined at training time; adding new tools requires retraining (vs ReAct which just needs a new prompt)
  • In the modern landscape: Toolformer's ideas evolved into function calling APIs (OpenAI, Anthropic) and tool-use fine-tuning (Gorilla, NexusRaven)
  • In system design: cite Toolformer for "how do models learn to use tools" and ReAct for "how do agents use tools at inference"