Research map / Large language models / AI and society
Addressing "Documentation Debt" in Machine Learning: A Retrospective Datasheet for BookCorpus
Jack Bandy, Nicholas Vincent
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Compare BookCorpus's genre mix with BookCorpusOpen and Smashwords21 | BookCorpus · BookCorpus, BookCorpusOpen, and Smashwords21 | romance books: 26.1% (2880) vs. 18.0% (3314) vs. 16.0% (66083) | Mixed |
| Compare BookCorpus's genre mix with BookCorpusOpen and Smashwords21 | BookCorpus · BookCorpus, BookCorpusOpen, and Smashwords21 | fantasy books: 13.6% (1502) vs. 17.2% (3171) vs. 10.6% (44032) | Mixed |
| Count unique books rather than total book files | BookCorpus · BookCorpus | unique books: 11,038 total; 7,185 unique | Mixed |
Closest papers by meaning
- The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track (NeurIPS 2024 · similarity 0.85)
- Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on HuggingFace (ICLR 2024 · similarity 0.84)
- Bridging the Data Provenance Gap Across Text, Speech, and Video (ICLR 2025 · similarity 0.83)
- BooookScore: A systematic exploration of book-length summarization in the era of LLMs (ICLR 2024 · similarity 0.82)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.82)
- Croissant: A Metadata Format for ML-Ready Datasets (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
