AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

How Text Quality Interventions Reshape Neural Scaling Laws for LLMs: Empirical Study

Newsha Ardalani, Feiyang Kang, Michael Kuchnik, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Carole-Jean Wu

ICLR 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add exact deduplication (ablation)QualityPajama CC data · compute scaling lawcompute efficiency: 100× gain in compute efficiency with data volume reduced to 83%Improved
use fuzzy dedup threshold 0.7 instead of 0.9 (ablation)dedupe_0.7 · compute scaling lawcompute required: 3× less compute than dedupe_0.9Improved
use fuzzy dedup threshold 0.7 instead of exact dedup (ablation)dedupe_0.7 · compute scaling lawcompute required: 10× less than exact deduplication, 300× less than no deduplicationImproved
filter strictly to high-PageRank pages (ablation)PageRank filtering · compute scaling lawcompute efficiencyNo clear effect
include pages not found in PageRank table (no_pr) (ablation)PageRank filtering · compute scaling lawcompute efficiencyImproved
mix synthetic and natural data instead of either alone (ablation)synthetic-natural mixing · compute scaling lawprojected lossImproved
add NSFW filtering on top of perplexity filter (ablation)high_perplexity · scaling exponent alphaalpha: 0.3509 -> 0.4341Improved
change from high_perplexity to nsfw filtered data (ablation)scaling law fit · irreducible loss EE: 0.35 -> 0.16Improved
use heuristic filters vs synthetic data for ranking consistencyscaling components · validation-set rank consistencySpearman correlation of A: 0.45 vs 0.81Mixed
vary data quality interventioncompute-optimal design · token-to-parameter ratioratio spread: up to 182× for the token-to-parameter ratioMixed
relate Zipf exponent of data to model exponentfiltered datasets · scaling exponentscorrelation with alpha: -0.37 with alpha, -0.005 with betaNo clear effect
use exact vs fuzzy dedup threshold 1.0 vs aesthetic baseline (ablation)deduped_1.0 · irreducible loss EE: 0.50 -> 0.05Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy