Research map / Large language models / Reasoning and chain of thought

Convex Dataset Valuation for Post-Training

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Process reward models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a convex, redundancy-aware dataset valuation method for LLM post-training based on gradient-space kernel mean matching (KMM), evaluated on multilingual SFT and GRPO transfer.

Problem

Selecting and weighting auxiliary datasets for LLM post-training under budget constraints, where per-dataset gradient alignment scores ignore redundancy among datasets and previews of data are limited.

Approach

Relaxes dataset selection to signed continuous weights and solves a convex QP (KMM in gradient space / Gram space) that rewards alignment with the target gradient and penalizes redundancy among auxiliary gradients, with an l1 budget constraint.

Key results

• TV+KMM reaches 46.24 best-k on Danish MMMLU-style SFT versus 46.05 for TV alone, and 34.60 on Marathi with the highest Borda score of 33.5.
• On Thai GRPO, One-Step+KMM achieves best-k 41.6 versus 39.2 for One-Step.
• TV+KMM is the only surrogate with a statistically significant rank correlation with full-training utility (Spearman 0.396, p=0.003) over 56 Danish subsets; KMM is over 100x cheaper than DataModel approaches.

Surprising findings

• KMM weights are allowed to be negative and compress magnitudes; seemingly unrelated languages such as Portuguese and Italian rise in rank for Marathi.
• Diagonal empirical-Fisher curvature variant performed slightly but consistently worse than the Euclidean KMM metric.

Limitations

• Advantage disappears when auxiliary data dominates the target signal (many auxiliary examples or very small target weight).
• Valuation scores are task-, model-, and metric-dependent and not intrinsic dataset values.
• Evaluation is mainly on multilingual tasks with a relatively small number of candidate datasets (N up to 25).

Experimental setup

Datasets: Aya Dataset Big-Math GSM8K split IFEval-related instruction-tuning auxiliaries (math, code, general)

Benchmarks: MMMLU MGSM IFEval MMLU-Malayalam

Metrics: best-k accuracy Borda count average performance Spearman correlation with full-training utility cross entropy

Baselines: One-Step gradient Task Vectors DataModel DataModel-CS GradEx-FS GradEx-RE Random

Builds on: kernel mean matching LESS task vectors GRPO DataModels

Closest papers by meaning

• What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions (NeurIPS 2025 · similarity 0.87)
• SAVA: Scalable Learning-Agnostic Data Valuation (ICLR 2025 · similarity 0.87)
• Limits to scalable evaluation at the frontier: LLM as judge won’t beat twice the data (ICLR 2025 · similarity 0.86)
• Reward-Augmented Data Enhances Direct Preference Alignment of LLMs (ICML 2025 · similarity 0.86)
• Compute-Constrained Data Selection (ICLR 2025 · similarity 0.86)
• Larger or Smaller Reward Margins to Select Preferences for LLM Alignment? (ICML 2025 · similarity 0.86)