Research map / Large language models / Instruction and fine-tuning
RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection
Yixin Yang, Qingxiu Dong, Linli Yao, Fangwei Zhu, Weilin Luo, Bin Wang, Zhifang Sui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on 15% RICO-selected Alpaca instead of full data | LLaMA3.1-8B · 12-benchmark avg | average score: 37.95 -> 43.37 | Improved |
| train on 15% RICO-selected Alpaca instead of full data | Qwen2.5-3B · 12-benchmark avg | average score: 46.65 -> 47.89 | Improved |
| train on RICO-selected Alpaca subsets instead of full data | LLaMA2-7B · 12-benchmark avg | average score: 38.54 -> 39.19 (1%); 38.45 at 15% | No clear effect |
| select with RICO instead of LESS | LLaMA3.1-8B · Avg of benchmarks, 15% | average score: 41.89 -> 43.37 | Improved |
| select with RICO instead of LESS | LLaMA3.1-8B · Pairwise winning score | pairwise winning score: 1.0019 | No clear effect |
| increase RICO-selected data beyond 15% (ablation) | LLaMA3.1-8B · Alpaca average score | average score | Hurt |
| apply Alpaca-trained RICO selector to WizardLM-70K (5%) | LLaMA3.1-8B · WizardLM avg | average score: 42.94 -> 47.48 | Improved |
| keep 15% RICO subset of WizardLM | LLaMA3.1-8B · WizardLM avg | average score: 42.94 -> 44.18 | Improved |
| add continuous refined score instead of binary labels (ablation) | RICO · Avg benchmarks (1% Alpaca) | average score | Improved |
| add fairness adjustment for length bias (ablation) | RICO · Avg benchmarks (1% Alpaca) | average score | Improved |
| RICO selection vs choosing hardest (Top PPL) samples | LLaMA3.1-8B · Avg of benchmarks, 15% | average score: 36.80 -> 43.37 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- SelectIT — SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection (NeurIPS 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.91)
- NICE Data Selection for Instruction Tuning in LLMs with Non-differentiable Evaluation Metric (ICML 2025 · similarity 0.90)
- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.89)
- SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection (NeurIPS 2024 · similarity 0.89)
- PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (ICLR 2024 · similarity 0.89)
- DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
