Research map / Label-efficient and robust learning / Recognition and long-tail
From Neurons to Neutrons: A Case Study in Interpretability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains small attention-ablated transformers to regress nuclear observables (binding energy, charge radius, separation energies) from proton/neutron numbers and uses PCA, latent space topography and symbolic regression to recover liquid-drop and shell-model structure.
Problem
Whether mechanistic interpretability of neural networks trained on scientific data can extract human-interpretable domain knowledge (here nuclear physics concepts) beyond good predictions.
Approach
Reframes mechanistic interpretability from "how does the model predict?" to "what can the model tell us about the data?", showing that PCA of learned embeddings and penultimate-layer features of nuclear-property regressors reveal a helix, parity splits, the semi-empirical mass formula volume/pairing terms and shell-model magic-number structure.
Key results
- Symbolic expression built from penultimate-layer PCs reaches RMS around 4600 keV on binding energy versus 8000 keV for the SEMF over the same dataset.
- Penultimate-layer PC0 matches the SEMF volume term, PC6 matches the pairing term, and PC4 matches a shell-model staircase structure (magic numbers).
- Latent embeddings of Z and N form a helix in the first three PCs, and orderedness and parity measures correlate with generalization across train/validation splits.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use neural-network penultimate-layer features for symbolic regression instead of fitting the task directly | Binding-energy symbolic regression · Binding energy dataset without TMS values | RMS: Direct regression gives considerably worse predictions; neural-feature expression: around 4600 keV | Helped |
| Use the neural-feature symbolic expression instead of the SEMF | Binding-energy prediction · Binding energy dataset without TMS values | RMS: 8000 keV -> around 4600 keV | Helped |
| Train jointly on multiple observables instead of one observable at a time | Nuclear-property prediction models · Nuclear observables, 50% validation split | RMS | Helped |
Surprising findings
- Models trained with no weight decay show continuous but non-helical embedding structure, suggesting weight decay drives the helix.
- Neural features recovered a symbolic formula that outperforms the SEMF, while direct symbolic regression on the task did not reach comparable performance.
Limitations
- Extracted knowledge is not guaranteed to be interpretable; the recovered symbolic expression is less interpretable than the SEMF.
- No quantitative metric exists for interpretability of extracted knowledge.
- Results are a proof-of-concept from a small attention-ablated model on a single domain dataset.
Experimental setup
Datasets: Atomic Mass Evaluation (AME) Atomic Data and Nuclear Data Tables 99 (2013)
Benchmarks: nuclear binding energy regression (AME data)
Metrics: RMS error (keV for energies, fm for lengths) orderedness parity ratio R_P
Baselines: semi-empirical mass formula (SEMF) direct symbolic regression on the task
Builds on: modular arithmetic mechanistic interpretability PySR semi-empirical mass formula
Closest papers by meaning
- Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.87)
- Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery (ICML 2025 · similarity 0.86)
- Validating Mechanistic Interpretations: An Axiomatic Approach (ICML 2025 · similarity 0.86)
- The Computational Complexity of Circuit Discovery for Inner Interpretability (ICLR 2025 · similarity 0.85)
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability? (NeurIPS 2025 · similarity 0.85)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.