Research map / Label-efficient and robust learning / Noisy labels
AQuA: A Benchmarking Tool for Label Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SimiFeat rather than the other evaluated cleaning methods | Label error detection in AQuA · Synthetic label noise across benchmark datasets | weighted F1 for detecting injected label errors | Helped |
| Use Confident Learning rather than the other evaluated cleaning methods | Label error detection in AQuA · Synthetic label noise across benchmark datasets | weighted F1 for detecting injected label errors | Hurt |
| Clean training labels with SimiFeat, CINCER, or AUM instead of leaving labels uncleaned | Downstream classifiers in AQuA · Synthetic noise across benchmark datasets | downstream weighted F1 | No clear effect |
| Add class-dependent or uniform label noise | Downstream classifiers in AQuA · Several datasets, including Electric Devices, MIT-BIH, and Dry Bean | downstream weighted F1 | Mixed |
| Evaluate cleaning methods separately by data modality | Label error detection in AQuA · Tabular datasets | weighted F1 for detecting injected label errors | Mixed |
| Use uniform or asymmetric noise rather than instance-dependent or class-dependent noise | Label error detection methods in AQuA · Synthetic noise types across benchmark datasets | weighted F1 for detecting injected label errors | Helped |
| Evaluate with weighted F1 rather than accuracy | Comparisons of label error detection methods · All benchmark datasets | method ranking | Mixed |
| Use CINCER rather than no cleaning | Downstream classifier on CIFAR-10 · Uniform noise | downstream weighted F1: 63.2 -> 65.0 | Helped |
Closest papers by meaning
- Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty (NeurIPS 2026 · similarity 0.87)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.86)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.86)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.86)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.86)
- Revisiting Interpolation for Noisy Label Correction (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.