Research map / Multi-agent systems and games / LLM agents

Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Progress-Aware Distillation (ProD), which penalizes stagnant student trajectories via an adaptive margin derived from the student–teacher stagnation gap, added to SFT distillation of small language model agents.

Problem

SFT-distilled small language model agents suffer progress stagnation, getting trapped in unproductive reasoning-action loops during long multi-turn tool-use trajectories.

Approach

Generate student trajectories, measure per-turn stagnation (lexical ROUGE-L, semantic embedding similarity, or LLM judge), and use the teacher–student stagnation gap plus answer correctness as an instance-dependent margin in a SimPO-style loss combined with the SFT loss; optionally iterate.

Key results

• ProD improves in-domain accuracy by 4.7% on average over SFT distillation across 8 SLMs (0.6B-8B) and out-of-domain by 4.9%.
• On HotpotQA, Qwen3-4B with ProD reaches 70.8% accuracy versus 66.6% for SFT and 71.0% for the GPT-4.1 teacher.
• ProD reduces average interaction turns by 11.6% and tokens by 20.7% relative to SFT.

Surprising findings

• Full-parameter fine-tuning does not alleviate stagnation and for Qwen models can make it more severe than LoRA.
• DPO on-policy underperforms SFT on Qwen3-4B average accuracy (74.4 vs 74.2 is slightly above, but DPO is below ProD); repetition penalty at higher strengths degrades accuracy.

Limitations

• Stagnation metrics are heuristic and the gap depends on teacher behavior; absolute scores vary across tasks.
• Some model families (Llama3.2-1B/3B, Gemma) retain a noticeable stagnation gap after one ProD iteration.
• Experiments are limited to MATH, HotpotQA, Omni-MATH and 2WikiMultiHopQA with a single teacher (GPT-4.1).

Experimental setup

Datasets: MATH HotpotQA

Benchmarks: MATH HotpotQA Omni-MATH 2WikiMultiHopQA

Metrics: accuracy turn-level lexical stagnation (ROUGE-L) semantic stagnation judgment stagnation average turns average tokens

Baselines: SFT distillation Repetition Penalty Concise SFT DPO SimPO GPT-4.1 teacher

Builds on: SimPO CodeAct agent distillation (Kang et al. 2025) LoRA ReAct

Closest papers by meaning

• Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.91)
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (ICLR 2026 · similarity 0.88)
• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.87)
• RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code (ICLR 2025 · similarity 0.87)
• Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents (NeurIPS 2026 · similarity 0.87)
• UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.87)