Research map / Large language models / AI and society

Copycats: the many lives of a publicly available medical imaging dataset

NeurIPS 2024 · Large language models › AI and society · approach: Benchmarking and reproducibility practice · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Audits 30 widely used CV, NLP and medical imaging datasets and their copies on community platforms (Kaggle, HuggingFace), finding vague licenses, unstable hosting, heavy duplication and missing metadata.

Problem

Publicly available medical imaging datasets on community-contributed platforms often lack clear licenses, persistent identifiers, documentation, and metadata needed for responsible use, and are duplicated across platforms.

Approach

Systematic audit of hosting, licensing, citation, and documentation practices for top-cited CV, NLP and MI datasets, combined with a Kaggle/HuggingFace duplicate and data-card analysis, to argue for commons-based data stewardship roles on CCPs.

Key results

• Kaggle hosts 640 ISIC-related datasets totaling 2.35 TB, versus 38 GB for the original ISIC archive.
• Kaggle hosts 350 BraTS-related datasets and 141 HAM10000 duplicates; BraTS duplicates total about 7.3 TB.
• Less than half of the most-used medical imaging datasets on Papers with Code have a license, and most top CV datasets have no clear license or terms of use.

Measured effects reported in the paper

Surprising findings

• Even DRIVE, listed on Papers with Code as CC-BY-4.0, was confirmed by its creators to have no license specified.
• Kaggle's 'well-documented' usability score can be 100% for datasets that still have issues, such as 'never' as update frequency and vague provenance.

Limitations

• Based on a limited set of 30 of the most cited datasets, with qualitative perceptions in addition to quantitative evidence.
• Did not reach out to Kaggle or HuggingFace or conduct practitioner interviews.
• Duplicate disambiguation was partly manual and some datasets could not be disambiguated.

Experimental setup

Datasets: CIFAR-10 ImageNet CIFAR-100 MNIST SVHN CelebA Fashion-MNIST CUB-200-2011 Places STL-10 GLUE SST

Metrics: hosting permanence license presence duplicate count and cumulative storage size documentation field completeness patient split reporting

Builds on: FAIR principles Datasheets for Datasets Croissant Wikipedia commons-based governance

Closest papers by meaning

• CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models (NeurIPS 2024 · similarity 0.84)
• FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models (NeurIPS 2024 · similarity 0.83)
• OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detection (CVPR 2025 · similarity 0.83)
• A Systematic Review of NeurIPS Dataset Management Practices (NeurIPS 2024 · similarity 0.83)
• The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track (NeurIPS 2024 · similarity 0.82)
• Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on HuggingFace (ICLR 2024 · similarity 0.82)