Research map / Large language models / Agents, code and math
Better Datasets Start from RefineLab: Automatic Optimization for High-Quality Dataset Refinement
Xiaonan Luo, Yue Huang, Ping He, Xiangliang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ILP assignment instead of greedy at 0.25C | RefineLab · MMLU 10% pilot | coverage JSD: 0.252 -> 0.183 | Improved |
| use ILP assignment instead of greedy at 0.25C | RefineLab · MMLU 10% pilot | difficulty JSD: 0.328 -> 0.388 | Hurt |
| use ILP assignment instead of greedy at 0.5C | RefineLab · MMLU 10% pilot | coverage JSD: 0.190 -> 0.063 | Improved |
| use ILP assignment instead of greedy at 1.0C | RefineLab · MMLU 10% pilot | distractor diversity: 0.300 -> 0.331 | Improved |
| use ILP assignment instead of uniform budget split at 1.0C | RefineLab · MMLU 10% pilot | difficulty JSD: 0.412 -> 0.295 | Improved |
| increase budget from 0.25C to 1.0C | RefineLab · MMLU 10% pilot | coverage JSD: 0.183 -> 0.019 | Improved |
| refine GSM8K with RefineLab | GSM8K · GSM8K | performance spread of top models: 2.2% -> 11.8% | Improved |
| refine GSM8K with RefineLab | GSM8K · GSM8K | gap o4-mini vs Mistral-8x7B: 30.8% -> 49.4% | Improved |
| refine HumanEval with RefineLab | HumanEval · HumanEval | performance spread of top models: 0.6% -> 13.4% | Improved |
| refine GSM8K with RefineLab | GSM8K · GSM8K prompting variants | average performance range across LLMs: 8.4% -> 4.9% | Improved |
| fine-tune on RefineLab-refined data instead of original | Llama-3.1-8B · GSM8K 150-item test | accuracy: 58.7% -> 65.6% | Improved |
| fine-tune on RefineLab-refined data instead of original | Qwen-2-7B · GSM8K 150-item test | accuracy: 67.3% -> 72.7% | Improved |
Closest papers by meaning
- Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance (NeurIPS 2025 · similarity 0.88)
- Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models (NeurIPS 2024 · similarity 0.87)
- Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods (ICLR 2026 · similarity 0.86)
- Self Iterative Label Refinement via Robust Unlabeled Learning (NeurIPS 2025 · similarity 0.86)
- DataMan: Data Manager for Pre-training Large Language Models (ICLR 2025 · similarity 0.86)
- Beyond Prompt Engineering: A Reinforced Token-Level Input Refinement for Large Language Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
