AI Paper Atlas

Research map / Large language models / AI and society

DataPerf: Benchmarks for Data-Centric AI Development

Mark Mazumder, Colby Banbury, Xiaozhe Yao, Bojan Karlaš, William A Gaviria Rojas, Sudnya Diamos, Greg Diamos, Lynn He, Alicia Parrish, Hannah Rose Kirk, Jessica Quaye, Charvi Rastogi and 35 more

NeurIPS 2023 · Large language models › AI and society · approach: Benchmarking and reproducibility practice · 50 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Cleanlab outlier rejection instead of nested cross-fold selectionSelection for Speech baselines · English, 25 samplesmacro F1: 0.32 -> 0.49Improved
use Cleanlab instead of nested cross-fold selectionSelection for Speech baselines · Indonesian, 25 samplesmacro F1: 0.36 -> 0.37No clear effect
use DataScope Shapley approximation instead of random ordering for label cleaningDebugging for Vision · Debugging for Vision% of samples fixed to reach threshold: 51.75 -> 15.54Improved
buy uniformly from all sellers instead of from a single random sellerData Acquisition baselines · market instance 4accuracy: 0.679 -> 0.742Improved
dataset iteration (selection, augmentation) with fixed modelDCAI Roman numeral challenge · DCAIaccuracy: 64.4% -> 85.8%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy