Research map / Large language models / Agents, code and math
Scaling Agents via Continual Pre-training
Liangcai Su, Zhen Zhang, Guangyu Li, Zhuo Chen, Chenxi Wang, Maojia Song, Xinyu Wang, Kuan Li, Jialong Wu, Xuanzhong Chen, Zile Qiao, Zhongwang Zhang and 14 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add agentic continual pre-training | Qwen3-30B-A3B-Base · BrowseComp-en (SFT-A) | Pass@1: 26.9 -> 31.4 | Improved |
| add agentic continual pre-training | Qwen3-30B-A3B-Base · BrowseComp-en (SFT-C) | Pass@1: 24.5 -> 38.8 | Improved |
| add agentic continual pre-training | Qwen3-30B-A3B-Base · HLE (SFT-C) | Pass@1: 27.9 -> 28.9 | Improved |
| add agentic continual pre-training | Qwen3-30B-A3B-Base · GAIA (SFT-B) | Pass@1: 71.8 -> 72.8 | Improved |
| add second long-context stage to single-stage CPT (ablation) | AgentFounder · BrowseComp-en | Pass@1: 31.4 -> 35.5 | Improved |
| add second long-context stage to single-stage CPT (ablation) | AgentFounder · BrowseComp-zh | Pass@3: 50.5 -> 58.5 | Improved |
| add second long-context stage to single-stage CPT (ablation) | AgentFounder · GAIA | Pass@3: 81.6 -> 82.5 | Improved |
| add FAS data for CPT (ablation) | Qwen3-30B-A3B-Base · BrowseComp-zh | Pass@1: 29.8 -> 37.0 | Improved |
| add HAS data to FAS data (ablation) | AgentFounder CPT · BrowseComp-zh | Pass@1: 37.0 -> 40.1 | Improved |
| add HAS data to FAS data (ablation) | AgentFounder CPT · GAIA | Pass@1: 72.8 -> 69.9 | Hurt |
| add HAS data to FAS data (ablation) | AgentFounder CPT · BrowseComp-en | Pass@1: 31.4 -> 31.4 | No clear effect |
| scale model size from 1B to 4B to 30B (ablation) | AgentFounder · deep research benchmarks (average) | accuracy: 20.4% -> 32.7% -> 48.9% | Improved |
Closest papers by meaning
- Offline Training of Language Model Agents with Functions as Learnable Weights (ICML 2024 · similarity 0.91)
- AgentRefine: Enhancing Agent Generalization through Refinement Tuning (ICLR 2025 · similarity 0.89)
- ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering (ICML 2026 · similarity 0.88)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.87)
- How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.87)
- Position: Foundation Agents as the Paradigm Shift for Decision Making (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
