Research map / Large language models / AI and society
Unreproducible Research is Reproducible
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a different random seed (single-seed comparison) (ablation) | 10 CNN models · MNIST, SVHN, EMNIST | model ranking | Hurt |
| use a different random seed (ablation) | 10 CNN models · CIFAR10, CIFAR100, TinyImageNet | model ranking | Hurt |
| tune hyper-parameters per dataset (unbiased) vs fixed He et al. settings (biased) (ablation) | 10 CNN models · six image datasets | distribution overlap of test error | No clear effect |
| fixed hyper-parameters from He et al. favoring PreActResNets (ablation) | PreActResNet18/101 · CIFAR10 | test error | No clear effect |
| PreActResNet vs ResNet (ablation) | PreActResNet · unbiased aggregated | ranking | Helped |
| PreActResNet vs ResNet (ablation) | PreActResNet · CIFAR10 | ranking | Hurt |
| PreActResNet vs ResNet (ablation) | PreActResNet · CIFAR100, TinyImageNet | ranking | No clear effect |
| larger depth ResNet101/PreActResNet101 vs 18 (ablation) | ResNet, PreActResNet · six image datasets | ranking | No clear effect |
| larger depth VGG19 vs VGG11 (ablation) | VGG · six image datasets | ranking | Helped |
Closest papers by meaning
- Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods (AAAI 2024 · similarity 0.88)
- What Do Machine Learning Researchers Mean by “Reproducible”? (AAAI 2025 · similarity 0.87)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.86)
- The Unreasonable Effectiveness of Open Science in AI: A Replication Study (AAAI 2025 · similarity 0.85)
- Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable (ICML 2025 · similarity 0.84)
- Position: Let’s Strengthen Verifiability if We Can’t Enforce Reproducibility (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.