Research map / Label-efficient and robust learning / Recognition and long-tail
An Information-Theoretic Criterion for Efficient Data Synthesis
Hanyu Li, Zhengqi Sun, Xiaotie Deng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the judge iteratively on its own outputs with no external rubric anchor | Implicit-rubric judge model · Instruction-following judging | acceptance rate | Hurt |
| use a binary all-rubrics-satisfied reward (coarse signal) despite a noisy judge | Rubric-based RL for instruction following · Out-of-distribution generalization | generalization: judge consistency 24% | Improved |
| rebalance positive/negative sources so length and style are decorrelated from correctness | Math/code/logic judge model · Balanced held-out correctness set | judging accuracy | Improved |
| assign all positives to one model family and all negatives to another (spurious length/style signal) | Math/code/logic judge model · Balanced held-out correctness set | correctness accuracy | Hurt |
| train judge with binary correctness on math only (high meta-level signal) | JudgeRLVR judge · Code, logic, general reasoning transfer | cross-domain judging | Improved |
| add mastery-gated length penalty to RL (math only training) | CoT compression RL · Code, instruction following, general QA | reasoning length | Improved |
| separate a fixed evaluator agent from the generator agent | Long-running agentic application harness · Agentic app development | overall quality | Improved |
Closest papers by meaning
- Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective (ICLR 2025 · similarity 0.88)
- Training on the Test Task Confounds Evaluation and Emergence (ICLR 2025 · similarity 0.87)
- A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops (ICLR 2025 · similarity 0.87)
- Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization (NeurIPS 2026 · similarity 0.87)
- Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World (ICML 2025 · similarity 0.86)
- Understanding Emergent Abilities of Language Models from the Loss Perspective (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
