Research map / Large language models / Language, safety and interpretability

Benford’s Curse: Tracing Digit Bias to Numerical Hallucination in LLMs

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Task vectors and steering

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that LLMs overgenerate small digits in a uniform-digit numerical benchmark, traces the bias to highly digit-selective FFN neurons in late layers, and shows that pruning a tiny set of digit-1 neurons corrects some erroneous outputs.

Problem

LLMs fail on basic numerical problems; it is unclear whether skewed (Benford-like) digit distributions in pretraining corpora induce digit generation bias that contributes to numerical hallucination.

Approach

Connects Benford's-law digit skew in the OLMo-mix-1124 pretraining corpus to model output bias, using a Digit Bias Benchmark with uniform ground-truth digits, logit-lens tracing, a Digit Selectivity Score for FFN neurons, and neuron pruning as a causal probe.

Key results

• Digit 1 frequency in model outputs often exceeds 12% on the uniform-digit benchmark, with digits 8 and 9 underrepresented (Mistral-7B).
• Pruning the top 0.01% digit-1-selective neurons corrects 0.19%-5.05% of erroneous outputs across four multistep tasks and four models, and reduces digit-1 output frequency.
• Aggregated neuron digit-selectivity profile correlates with pretraining digit frequency at Pearson r = 0.949.

Measured effects reported in the paper

Surprising findings

• First-error digits show an even stronger skew toward small digits than overall outputs, closely following Benford's Law.
• Pruning digit-1-selective neurons corrects a few erroneous outputs in GSM8K and SVAMP while potentially harming some correct predictions.

Limitations

• Does not establish causality between corpus digit frequency and model bias; no controlled pretraining interventions.
• Experiments limited to 7B-9B dense models with standard MLPs; MoE and larger models untested.
• Pruning is coarse and may disrupt correct predictions, limiting accuracy gains.

Experimental setup

Datasets: OLMo-mix-1124 DeepMind Mathematics Dataset GSM8K SVAMP

Benchmarks: Digit Bias Benchmark GSM8K SVAMP

Metrics: digit output frequency first-error digit distribution corrected proportion after pruning Spearman correlation of DSC Pearson correlation with corpus digit frequency

Builds on: logit lens Benford's Law Mathematics Dataset key-value memory view of FFN (Geva et al.)

Closest papers by meaning

• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.88)
• Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference (NeurIPS 2025 · similarity 0.87)
• Do LLMs Signal When They’re Right? Evidence from Neuron Agreement (ICML 2026 · similarity 0.86)
• Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.86)
• Controlling Equational Reasoning in Large Language Models with Prompt Interventions (AAAI 2025 · similarity 0.85)
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.85)