Research map / Large language models / Instruction and fine-tuning
Agent Skill Acquisition for Large Language Models via CycleQD
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CycleQD, a MAP-Elites-style evolutionary merging method that cyclically swaps task metrics between quality and behavior characteristics, with model-merging crossover and SVD-based mutation, to combine Llama3-8B-Instruct experts for coding, OS, and DB agent skills.
Problem
Training LLMs to acquire multiple agent skills suffers from data-ratio imbalance and next-token objectives that poorly match task performance; the paper seeks a way to merge single-task experts into one model without data-ratio or objective tuning.
Approach
Runs MAP-Elites-style quality diversity where each task's metric is alternately the quality measure and the behavior characteristics (cyclic swap), uses task-vector merging as crossover and SVD-component-aligned perturbations as mutation, and aggregates archive elites via softmax-weighted task vectors.
Key results
- CycleQD averaged 52.4 across MBPP, DB, and OS, versus 51.6 for the best merging baseline (NSGA-II) and 47.0 for fine-tuning on all data; gpt-3.5-turbo averaged 53.7.
- On the 4-task VQA+CS setup, CycleQD reached 49.7 average versus 34.2 for the base model and 45.5 for the best expert.
- On SAM merging, top model pairs (CAM+POL, POL+SKL, CAM+SKL) retained 0.92–0.97 of expert-normalized average scores; similarity correlated 0.83 with merge quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CycleQD instead of the strongest merging baseline | Llama3-8B-Instruct · MBPP | pass@1: 75.9 -> 76.4 | Helped |
| Apply CycleQD instead of the strongest baseline | Llama3-8B-Instruct · DB | success rate: 42.4 -> 38.2 | Hurt |
| Apply CycleQD instead of the strongest baseline | Llama3-8B-Instruct · OS | success rate: 42.6 -> 42.6 | No clear effect |
| Apply CycleQD instead of the strongest baseline | Llama3-8B-Instruct · AgentBench CS tasks | Avg: 51.6 -> 52.4 | Helped |
| Add cyclic quality and BC alternation (ablation) | QD · AgentBench CS tasks | Avg: 47.6 -> 49.4 | Helped |
| Add Gaussian mutation (ablation) | CycleQD with no mutation and random sampling · AgentBench CS tasks | Avg: 49.4 -> 48.5 | Hurt |
| Replace Gaussian mutation with SVD mutation (ablation) | CycleQD with random sampling · AgentBench CS tasks | Avg: 48.5 -> 51.7 | Helped |
| Add Elite sampling instead of random sampling (ablation) | CycleQD with SVD mutation · AgentBench CS tasks | Avg: 51.7 -> 52.4 | Helped |
| Add SVD mutation (ablation) | CycleQD with random sampling · AgentBench CS tasks | Avg: 49.4 -> 51.7 | Helped |
| Merge expert models with CycleQD | SAM-ViT Huge · CAM + POL segmentation | Avg Score | Helped |
| Merge expert models with CycleQD | SAM-ViT Huge · CAM + LEA segmentation | Avg Score | Mixed |
Surprising findings
- The merged model's average of 52.4 nearly matches gpt-3.5-turbo (53.7) with an 8B open-weight model.
- Naive Gaussian mutation hurt performance relative to no mutation (48.5 vs 49.4), while SVD-aligned mutation improved it.
Limitations
- Merging success depends on compatibility of source models; highly divergent experts merge poorly (SAM CAM+LEA and POL+LEA).
- Hyperparameters chosen from arbitrary preliminary trials without systematic search.
- Higher compute cost (410 H100 GPU hours) than fine-tuning baseline, mostly from agentic evaluation.
Experimental setup
Datasets: MBPP+ AgentBench OS AgentBench DB Magicoder-Evol-Instruct-110K Magicoder-OSS-Instruct-75K Agent-FLAN TextVQA COD10K CAMO CHAMELEON Kvasir CVC-ClinicDB
Benchmarks: MBPP+ AgentBench OS AgentBench DB HUMANEVAL+ BigCodeBench MMLU BBH GSM8K SQuAD2 TriviaQA HellaSwag OpenBookQA
Metrics: pass@1 success rate normalized score IoU-style segmentation score (normalized to expert)
Baselines: Fine-tuning (All) Merging w/o learning (averaging) Merging learning w/ GD Merging learning w/ CMA-ES Merging learning w/ NSGA-II gpt-3.5-turbo gpt-4
Builds on: MAP-Elites task arithmetic model merging NSGA-II AgentBench Agent-FLAN
Closest papers by meaning
- AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks (NeurIPS 2025 · similarity 0.89)
- Fleet of Agents: Coordinated Problem Solving with Large Language Models (ICML 2025 · similarity 0.87)
- Hybrid-Gym: Training Coding Agents to Generalize Across Tasks (ICML 2026 · similarity 0.86)
- Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning (NeurIPS 2024 · similarity 0.86)
- A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks (NeurIPS 2026 · similarity 0.86)
- Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.