AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity

Yeongbin Seo, Gayoung Kim, Jaehyung Kim, Jinyoung Yeo

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
filter with prior-based method instead of PPL-based filteringGPT-2 1.5B pretraining on Dolma · Dolma, 20 benchmarks avgavg normalized accuracy: 8.22 -> 9.20Improved
filter with prior-based method instead of PPL-based filteringGPT-2 137M pretraining on Dolma · Dolma, 20 benchmarks avgavg normalized accuracy: 5.26 -> 6.65Improved
use prior-based filter instead of PPL-based for filtering costdata filtering of 6B-token Dolma · Dolma filtering timefiltering time: 216 GPU hrs -> 0.25 hrsImproved
filter with PPL-based method rather than no filter on symbolic problem solvingGPT-2 1.5B on Dolma · Dolma, symbolic problem solvingnormalized accuracy: 13.22 -> 7.91Hurt
filter with prior-based method on code corpus instead of PPL-basedGPT-2 1.5B on Pile-github · Pile-github, 6 symbolic benchmarksavg normalized accuracy: 11.21 -> 12.03Improved
filter with prior-based method on code corpus instead of PPL-basedGPT-2 137M on Pile-github · Pile-github, 6 symbolic benchmarksavg normalized accuracy: 9.82 -> 12.19Improved
PPL-based filtering on code corpus versus no filtering at 137MGPT-2 137M on Pile-github · Pile-github, 6 symbolic benchmarksavg normalized accuracy: 10.15 -> 9.82No clear effect
use prior statistics from 1% subset of corpus instead of full corpus (ablation)prior-based filter · Dolma outlier set overlapoutlier overlap with full-corpus setNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy