Research map / Architectures and efficiency / RNNs and normalization
Identifying Statistical Bias in Dataset Replication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Correct for noisy selection-frequency matching with a parametric model | ImageNet-v2 accuracy-gap estimate · ImageNet-v1 vs ImageNet-v2 | accuracy gap: 11.7% ± 1.0% -> 3.6% ± 1.5% | Helped |
| Use the naive estimator with remeasured selection frequencies | ImageNet-v2 · ImageNet-v1 vs ImageNet-v2 | adjusted accuracy gap: 11.7% -> 5.7% | Helped |
| Apply the jackknife to the naive estimator (ablation) | ImageNet-v2 adjusted accuracy estimate · ImageNet-v1 vs ImageNet-v2 | adjusted accuracy gap: 5.7% -> 4.6% | Helped |
| Reduce the number of matching annotators from 10 to 5 (ablation) | ImageNet-v2 replication sampling · ImageNet-v1 vs ImageNet-v2 | accuracy gap: 12% -> 14% | Hurt |
| Increase matching annotators per candidate image (ablation) | ImageNet-v2 replication sampling · ImageNet-v1 vs ImageNet-v2 | replication accuracy: 12.3% at n = 10; the gap decreases as n grows | Helped |
| Filter the source candidate pool using held-out selection-frequency data (ablation) | Statistic-matched replication · ImageNet-v1 vs ImageNet-v2 | accuracy: by ~3% | Helped |
| Clean the annotation data (ablation) | Remeasured ImageNet-v1 and ImageNet-v2 selection frequencies · ImageNet-v1 vs ImageNet-v2 | selection-frequency gap: 4.5% -> 4.6% | Mixed |
| Use the jackknife correction instead of the naive estimator (ablation) | ImageNet-v2 adjusted accuracy estimate · ImageNet-v1 vs ImageNet-v2 | adjusted accuracy gap: 5.7% -> 4.6% | Helped |
Closest papers by meaning
- Ameliorate Spurious Correlations in Dataset Condensation (ICML 2024 · similarity 0.87)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.87)
- Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods (AAAI 2024 · similarity 0.85)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.85)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.84)
- An Effective Theory of Bias Amplification (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.