Research map / Large language models / AI and society

Position: Application-Driven Innovation in Machine Learning

ICML 2024 · Large language models › AI and society · approach: Benchmarking and reproducibility practice

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that application-driven machine learning (ADML) research is undervalued and proposing changes to reviewing, hiring, and teaching practices to support it.

Problem

Application-driven ML research is systematically under-valued in the ML community because reviewing, hiring, and teaching norms are aligned with methods-driven research centered on standardized benchmarks.

Approach

Frames application-driven ML (ADML) as a paradigm distinct from methods-driven research (real-world tasks, application-specific metrics, auxiliary domain knowledge, problem-informed methods) and argues it contributes to core ML, then gives concrete recommendations for reviewers, hiring committees, and educators.

Key results

• Reports that three standard benchmarks (ImageNet, MS COCO, OpenAI Gym) were cited a total of 26,000 times in 2023, illustrating benchmark hegemony.
• Notes that in many remote sensing problems a simple random forest outperforms Vision Transformers, illustrating gaps between methods-driven expectations and real-world performance.

Surprising findings

• Fourier Neural Operators, developed for fluid-flow PDEs, have been reused in climate data superresolution and materials property prediction, suggesting real-world challenges share more structure than artificial benchmarks.

Limitations

• No empirical evaluation; claims rest on argument and examples.
• Does not address how applications are chosen or whose values determine what counts as a solution.

Experimental setup

Builds on: U-Net physics-informed neural networks Fourier Neural Operators

Closest papers by meaning

• Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods (AAAI 2024 · similarity 0.84)
• Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption (ICML 2025 · similarity 0.84)
• AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench (NeurIPS 2025 · similarity 0.83)
• Learning About Algorithm Auditing in Five Steps: Scaffolding How High School Youth Can Systematically and Critically Evaluate Machine Learning Applications (AAAI 2025 · similarity 0.83)
• MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.83)
• Task-Agnostic Machine-Learning-Assisted Inference (NeurIPS 2024 · similarity 0.82)