Research map / Label-efficient and robust learning / Recognition and long-tail
Mitigating dataset harms requires stewardship: Lessons from 1000 papers
Kenneth L Peng, Arunesh Mathur, Arvind Narayanan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Retract the parent dataset without addressing remaining copies, derivatives, or models | MS-Celeb-1M · 2020 research papers | documented uses: 54 | Mixed |
| Retract the parent dataset without addressing remaining copies, derivatives, or models | DukeMTMC · 2020 research papers | documented uses: 73 | Mixed |
| Create derivatives that retain original images and remain available after parent retraction | MS-Celeb-1M · 2020 research papers | uses through derivative datasets or pre-trained models: 17; 26 | Mixed |
| Create derivatives that retain original images and remain available after parent retraction | DukeMTMC · 2020 research papers | uses through derivative datasets: 72 of 73 | Mixed |
| Make retraction reasons and requests not to use the data explicit and visible | Dataset retraction · Retracted datasets | continued use | Improved |
| Make license restrictions explicit about whether they cover trained models and derivatives | Non-commercial datasets · Production use | commercially licensed model releases: 18 | Improved |
| Use a non-commercial license without specifying its implications for pre-trained models | MS-Celeb-1M · GitHub repositories containing pre-trained models | repositories including the same non-commercial designation: 3 of 21 | Hurt |
| Allow technological progress to make a research benchmark viable for production applications | LFW · LFW website benchmark results | commercial results: 3 of 38; 41 of 49 | Mixed |
| Use dataset derivatives to enable new applications, add sensitive annotations, or process images | LFW, DukeMTMC, and MS-Celeb-1M derivatives · 41 identified derivatives | derivatives with new applications: Twenty-one of 41 | Mixed |
| Rely on conventional citation practices rather than stable identifiers and dataset-specific tracking | Dataset research · Sampled papers and citation cases | citations unable to be disambiguated: 42 of 446 | Hurt |
Closest papers by meaning
- A Systematic Review of NeurIPS Dataset Management Practices (NeurIPS 2024 · similarity 0.86)
- A Taxonomy of Challenges to Curating Fair Datasets (NeurIPS 2024 · similarity 0.83)
- Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on HuggingFace (ICLR 2024 · similarity 0.82)
- The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track (NeurIPS 2024 · similarity 0.82)
- MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation from an Information-theoretic Perspective (ICML 2024 · similarity 0.82)
- Stop the Nonconsensual Use of Nude Images in Research (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
