Research map / Large language models / AI and society
Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that deep learning phenomena (double descent, grokking, lottery ticket hypothesis) are edge cases with limited practical relevance, and that research on them should be pursued pragmatically to refine broad explanatory theories rather than producing narrow ad hoc hypotheses.
Problem
Research on surprising deep learning phenomena often produces narrow, case-by-case explanations that have limited practical relevance and broader utility for the field.
Approach
Distinguishes narrow ad hoc hypotheses from broad explanatory theories and argues that edge-case phenomena are valuable mainly as test beds for refining broad theories, guided by a pragmatic, sociotechnical view of research value and rigorous scientific practice.
Key results
- Citation count of the three original phenomenon papers is 7272 (as of 5 June 2025), and 108-149 main-track papers per conference at NeurIPS 2024, ICML 2024, and ICLR 2024 mention these phenomena.
- A prime-number-count illustrative experiment reproduces double descent and grokking-like curves, which the authors argue is an artifact of parameter magnitude distributions rather than meaningful insight.
Surprising findings
- Lottery ticket hypothesis originator Jonathan Frankle reportedly stated people should not be working on lottery tickets as a research area.
Limitations
- Selection of three phenomena is non-exhaustive and judgments of practical irrelevance are subjective
- Distinction between narrow and broad explanations involves subjectivity
- Acknowledged that the position is not entirely new
Experimental setup
Builds on: Popper's falsifiability sociotechnical pragmatism
Closest papers by meaning
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.87)
- Proving the Lottery Ticket Hypothesis for Convolutional Neural Networks (ICLR 2022 · similarity 0.85)
- Position: A Critical Perspective on The Value in Studying Deep Learning Phenomena (ICML 2025 · similarity 0.84)
- Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond (NeurIPS 2024 · similarity 0.84)
- Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ (Abstract Reprint) (AAAI 2024 · similarity 0.84)
- Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.