Research map / Large language models / Instruction and fine-tuning

Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation

ICML 2025 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Adapting While Learning (AWL), a two-stage fine-tuning method that internalizes tool-generated scientific knowledge (WKL) and trains the model to answer easy problems directly while switching to tools for hard ones (TUA), evaluated on six scientific benchmarks with Llama-3.1-8B-Instruct.

Problem

LLMs hallucinate on scientific problems, and tool-augmented fine-tuning makes them overreliant on tools, incurring unnecessary cost; they lack adaptive decisions about when to use tools.

Approach

Train the model on tool-generated solutions so knowledge is internalized (WKL), then partition questions into easy/hard by the model's direct-answer accuracy and keep direct-answer targets for easy questions while using tool-trace targets for hard ones (TUA), combined in a single mixed loss to avoid prompt inconsistency.

Key results

• AWL-trained Llama3.1-8B with intelligent tool use (P_i) reaches 66.13% average answer accuracy vs 56.16% for the base model with P_i, a reported 29.11% average improvement over the base model.
• Tool usage accuracy averages 63.93% for AWL vs 51.21% for the base model, a reported 12.72% improvement.
• On open-ended Epidemiology tasks, AWL-RL-P_i achieves 56.25% meeting requirements vs 43.75% for GPT-4o, and reduces tool calls from 2.80 to 0.42 per question.

Measured effects reported in the paper

Surprising findings

• Naive alternation between WKL and TUA loss terms caused significant performance drops on easy problems, attributed to prompt inconsistency.
• The base model without training over-relies on tools (tool usage accuracy near 50%), while closed models such as GPT-4o and Claude-3.5 often answer directly on public datasets, possibly due to pretraining exposure.

Limitations

• Requires domain-specific fine-tuning; no unified training across scientific domains.
• Binary easy/hard partition ignores the spectrum of problem complexity.
• Remaining errors are dominated by agent limitations for multi-step tool interactions with the 8B base model.

Experimental setup

Datasets: MATH SciBench Mujoco (custom) PDE (custom) Climate (custom) Epidemiology (custom)

Benchmarks: MATH SciBench Mujoco PDE Climate Epidemiology

Metrics: answer accuracy tool usage accuracy tool usage rate proportion of proposals meeting threshold

Baselines: Llama3.1-70B GPT-4o GPT-4o-mini Claude-3.5-Sonnet Llama3.1-8B-Instruct (base) in-context learning tool-use baseline Qwen2.5-14B-Instruct

Builds on: direct preference optimization Llama-Factory

Closest papers by meaning

• Automated Creation of Reusable and Diverse Toolsets for Enhancing LLM Reasoning (AAAI 2025 · similarity 0.90)
• SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding (ICLR 2025 · similarity 0.88)
• Large Language Models as Tool Makers (ICLR 2024 · similarity 0.88)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models (NeurIPS 2024 · similarity 0.87)
• From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions (ICLR 2025 · similarity 0.87)
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models (ICML 2024 · similarity 0.87)