Research map / Detection and segmentation / Open-set and few-shot detection
HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions
Lingjiao Chen, Zhihua Jin, Sabri Eyuboglu, Christopher Re, Matei Zaharia, James Y. Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| API update 2020 to 2021 | IBM speech API · DIGIT | accuracy: +4% | Improved |
| API update 2020 to 2021 | Microsoft speech API · DIGIT | accuracy: 1.5 | Hurt |
| API update 2020 to 2021 | Google speech API · DIGIT | accuracy: 20 | Improved |
| API update 2020 to 2021 | IBM speech API · DIGIT | prediction overlap: 79.4 | Hurt |
| API update 2020 to 2021 | Microsoft speech API · DIGIT | confidence movement: 31.7 | Mixed |
| API update over time | Google speech API · DIGIT | group disparity (accent) | Hurt |
| API update 2020 to 2022 | EPixel API · COCO | accuracy: 20 | Hurt |
| API update 2020 to 2022 | Google API · COCO | accuracy: 1 | Improved |
| API update 2020 to 2022 | Microsoft API · COCO | group disparity (gender): 10 | Mixed |
| API update 2020 to 2022 | EPixel API · COCO | prediction overlap: 30 | Hurt |
| API update March to August 2022 | IBM NER API · GMB | accuracy: 50 -> 45 | Hurt |
| API update 2020 to 2022 | Amazon sentiment API · IMDB | accuracy: 10 | No clear effect |
Closest papers by meaning
- The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track (NeurIPS 2024 · similarity 0.83)
- Croissant: A Metadata Format for ML-Ready Datasets (NeurIPS 2024 · similarity 0.83)
- Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets (NeurIPS 2026 · similarity 0.83)
- CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets (NeurIPS 2026 · similarity 0.82)
- Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models (NeurIPS 2025 · similarity 0.82)
- Gorilla: Large Language Model Connected with Massive APIs (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
