Research map / Large language models / Instruction and fine-tuning
Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Star-Agents uses multiple LLM agent-pairs to rewrite instructions and responses, then selects data via a dual-model IFD-gap plus LLM-referee score, and evolves agent-pair sampling probabilities to instruction-tune Pythia-1B and Llama-2-7B.
Problem
Instruction tuning data quality and diversity are costly to obtain; single-LLM generation yields limited style variety and difficulty that may be too complex for small target models.
Approach
Pair multiple LLM agents for instruction and response rewriting, score samples with a dual-model IFD difference (small vs. large model) combined with an LLM-referee quality score, and evolve agent-pair sampling probabilities plus an instruction memory bank based on those scores.
Key results
- Pythia-1B-star_instruct averages 4.17 across Vicuna-Bench, WizardLM testset and MT-Bench, versus 3.73 for Pythia-1B-evol_instruct and 3.10 for Pythia-1B-alpaca.
- Llama-2-7B-star_instruct averages 6.95 versus 6.35 for Llama-2-7B-evol_instruct, and scores 5.74 on MT-Bench versus 5.21.
- The paper reports up to 40% gains on math, coding and Fermi problems and an average ~12% improvement over Evol-Instruct; ablation shows removing evolution drops average from 4.17 to 3.97.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Pythia-1B on Star Instruct instead of Evol-Instruct | Pythia-1B · Average across Vicuna-bench, WizardLM testset, and MT-Bench | score: 3.73 -> 4.17 | Helped |
| Train Llama-2-7B on Star Instruct instead of Evol-Instruct | Llama-2-7B · Average across Vicuna-bench, WizardLM testset, and MT-Bench | score: 6.35 -> 6.95 | Helped |
| Add the evolution strategy (ablation) | Star-Agents with diversity and data selection · Pythia-1B average score | score: 3.97 -> 4.17 | Helped |
| Add data selection (ablation) | Star-Agents with diversity · Pythia-1B average score | score: 3.62 -> 3.97 | Helped |
| Use diverse data with random sampling instead of baseline ChatGPT-generated data (ablation) | Pythia-1B · Average score | score: 3.73 -> 3.62 | Hurt |
| Use dual-model selection instead of random selection | Pythia-1B · Average score | score: 3.62 -> 4.17 | Helped |
| Use dual-model selection instead of IFD selection | Pythia-1B · Average score | score: 3.26 -> 4.17 | Helped |
| Prioritize agent-pairs using evolutionary sampling | Star-Agents · Agent-pair sampling probability | sampling probability: approximately 10% to 30%; near zero | Mixed |
Surprising findings
- Evol-Instruct data, though more complex, reduced performance for 14M and 70M parameter models compared with Alpaca, suggesting overly complex data can hurt small models.
- Diversified agent-generated data with random sampling performed slightly below the ChatGPT-generated baseline, attributed to uneven quality.
Limitations
- Evaluated only on single-turn instruction datasets; multi-turn conversations not evaluated.
- Domain-specific instruction validation left to future work.
- Relies on ChatGPT as referee and as a generator, which limits reproducibility and fairness of comparison.
Experimental setup
Datasets: Evol-Instruct Alpaca
Benchmarks: MT-Bench Vicuna-Bench WizardLM testset Open LLM Leaderboard (ARC, HellaSwag, MMLU, TruthfulQA)
Metrics: GPT-4 judge score average score
Baselines: Alpaca Evol-Instruct IFD Random selection OPT-1.3B Sheared-LLaMA-1.3B Llama-2-7B-alpaca sRecycled-Wiz-7B-v2 zephyr-beta-sft mpt-7B-chat XGen-7B-8k-Inst
Builds on: Evol-Instruct Cherry LLM (IFD) Self-Instruct Alpaca
Closest papers by meaning
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.89)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.88)
- Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning (ICML 2024 · similarity 0.88)
- MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching (NeurIPS 2025 · similarity 0.88)
- Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers (ICML 2024 · similarity 0.88)
- SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.