Research map / Probabilistic and scientific modeling / Molecules and proteins
Position: Is machine learning good or bad for the natural sciences?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that machine learning's ontology (only data exist) and epistemology (good if it predicts held-out data) clash with natural science, identifying where ML is useful (nuisance/confounder modeling, operations) and where it introduces biases (emulator confirmation bias, population-level bias from regression labels), illustrated by a toy MLP example.
Problem
Whether and where machine learning is good or bad for the natural sciences, given ML's data-centric ontology and predictive epistemology versus the latent-structure-focused aims of science.
Approach
Argues ML's ontology and epistemology should be matched to the role it plays in a scientific project: expressive ML is conservative for modeling nuisances and confounders, but harmful when emulators replace simulations (confirmation bias) or when regression-generated labels are used in population analyses (amplified bias).
Key results
- In the toy example, an MLP trained on 4096 points and applied to a 10^5-point test set yields population-level mean-age estimates biased away from truth at roughly the 30-sigma level, while the training-set binned means remain consistent with truth.
- The authors argue the emulator-induced confirmation bias and the population-level regression bias cannot be easily corrected in general, since validation data are usually insufficient to assess bias.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transfer MLP-estimated labels to a large test set and use them for population-level inference | Three-layer MLP regression · Toy data, population inference by guiding radius | bias: 30-sigma | Hurt |
| Use training-set labels instead of regression-transferred labels for population inference | Toy data analysis · Toy data, relationship between measured age and guiding radius | bias | Helped |
| Use an ML emulator in place of or alongside first-principles simulations | Simulation-based scientific inference · Natural-science simulations | confirmation bias | Hurt |
| Use expressive ML regressions to generate labels that are later combined in joint or ensemble analyses | ML-generated dataset labels · Population, joint, or ensemble analyses | estimator bias | Hurt |
| Use a more expressive model for confounders | Causal inference · Causal analyses with confounding inputs | robustness of causal conclusions | Helped |
Surprising findings
- Population-level averages of regression outputs can be highly biased and precise, even though individual-object validation shows no strong bias and test data match training distribution.
- Expressive ML models can make causal conclusions more conservative when used to model confounders, as flexible confounder models leave less room for spurious effects.
Limitations
- Argument is largely qualitative and perspective-driven, rooted in physics and astronomy.
- Toy example is simplified and does not test expressivity or model misspecification in depth.
- Proposed biases are asserted to lack fixes without a general demonstration across realistic pipelines.
Experimental setup
Datasets: synthetic toy data (stellar-spectrum-like feature vectors with age and guiding radius)
Metrics: binned mean of label versus guiding radius bias relative to truth
Baselines: multilayer perceptron regression on population averages
Builds on: Bayesian likelihood-based inference simulation-based inference
Closest papers by meaning
- Prediction-Augmented Trees for Reliable Statistical Inference (NeurIPS 2026 · similarity 0.82)
- Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.81)
- Position: The Causal Revolution Needs Scientific Pragmatism (ICML 2024 · similarity 0.81)
- Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption (ICML 2025 · similarity 0.81)
- Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track (NeurIPS 2025 · similarity 0.80)
- Task-Agnostic Machine-Learning-Assisted Inference (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.