Research map / Large language models / AI and society
Croissant: A Metadata Format for ML-Ready Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Croissant, a schema.org-based JSON-LD metadata format for ML datasets, and evaluates its usability with a human annotator study on ten datasets.
Problem
Working with ML datasets is hard because of heterogeneous formats, poor interoperability between tools, and documentation that is natural-language only and not machine-readable.
Approach
A machine-readable metadata vocabulary built on schema.org/Dataset with resource, structure (RecordSet), and semantic layers plus a Responsible AI extension, so datasets become loadable into ML frameworks without changing their underlying data layout.
Key results
- Over 80% of annotations (25 of 30) judged Croissant attributes to capture important dataset information.
- Average BLEU agreement was 0.55 for Croissant attributes versus 0.41 for Croissant-RAI attributes.
- Around 90% of annotations stated the specification was understandable; about 83% found attribute names intuitive.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Croissant attributes rather than Croissant-RAI attributes | Croissant metadata annotations · 10 annotated ML datasets | BLEU: 0.41 -> 0.55 | Helped |
| Use attributes with predefined values rather than free-form Text values | Croissant metadata annotations · Annotated attributes | BLEU: 0.4 -> 0.59 | Helped |
| Request Croissant-RAI attributes | Croissant metadata annotations · 30 annotations | annotations rating attributes redundant or not definable: around 23% | Hurt |
Surprising findings
- Croissant-RAI attributes such as annotator demographics and personal sensitive information were often missing from documentation, causing redundancy ratings.
- sc:creator and sc:publisher were commonly confused by annotators.
Limitations
- Small user study with nine annotators drawn from the Croissant community, possibly biased and not representative of non-expert users.
- BLEU is a noisy agreement metric for free-text attributes; Fleiss' Kappa not applicable.
- Croissant is not fully complete, and the editor and documentation are still early-stage.
Experimental setup
Datasets: MMLU Dolly-15k FLORES CIFAR10 MSCOCO Visual Genome MMMU MathVista MLS_Eng librispeech_asr
Metrics: BLEU agreement between annotators Likert ratings for consistency, completeness, conciseness, r
Builds on: schema.org Dataset vocabulary Datasheets for Datasets Data Cards TensorFlow Datasets
Closest papers by meaning
- Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets (NeurIPS 2026 · similarity 0.96)
- CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets (NeurIPS 2026 · similarity 0.93)
- The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track (NeurIPS 2024 · similarity 0.85)
- Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on HuggingFace (ICLR 2024 · similarity 0.84)
- A Systematic Review of NeurIPS Dataset Management Practices (NeurIPS 2024 · similarity 0.82)
- Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs (ICLR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.