Research map / Large language models / Language, safety and interpretability
Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Tests whether LLM in-context learning on exchangeable synthetic data satisfies the martingale property and Bayesian uncertainty scaling, finding violations for Llama-2-7B, Mistral-7B, GPT-3, GPT-3.5 and GPT-4.
Problem
Whether in-context learning in LLMs on exchangeable data is (approximately) Bayesian, tested via the martingale property and Bayesian scaling of epistemic uncertainty.
Approach
Uses the martingale property of predictive distributions, a necessary condition for Bayesian exchangeable learning, as a falsifiable test: checks invariance of predictions to imputed missing samples via sample paths, compared against bootstrap CIs from a reference Bayesian model, plus a scaling test of the martingale posterior's spread (IQR of MLE).
Key results
- In short-horizon Bernoulli checks (m=n/2) most LLMs stay within CIs, but with m=2n deviations become frequent; GPT-4 deviates for theta in {0.3, 0.5} at m=n/2.
- In the Gaussian experiment (theta=-1, n=100, m=50), all models except GPT-3.5 show clear deviation from the martingale property.
- Llama-2-7B and GPT-3.5 show slower epistemic uncertainty scaling than a Bayesian model; GPT-3.5 is overconfident in small samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the Bernoulli sampling horizon from m=n/2 to m=2n | LLMs on Bernoulli data · Bernoulli, θ∈{0.3, 0.5, 0.7} | martingale property adherence | Hurt |
| Use short sampling paths (m=n/2) | Most tested LLMs on Bernoulli data · Bernoulli, θ∈{0.3, 0.5, 0.7} | martingale property adherence | Helped |
| Use GPT-4 instead of the other tested LLMs | LLMs on Bernoulli data with short paths · Bernoulli, θ∈{0.3, 0.5}, m=n/2 | martingale property adherence | Hurt |
| Use GPT-3.5 instead of GPT-4 | GPT models on Bernoulli data · Bernoulli | martingale property adherence | Helped |
| Use GPT-3.5 instead of GPT-3.5's short-horizon setting | GPT-3.5 on Gaussian data · Gaussian, θ=-1, n=100, m=50 | martingale property adherence | Hurt |
| Use GPT-3.5 on the synthetic natural-language task | GPT-3.5 · Synthetic natural language, n=80, m=40 | martingale property adherence | Hurt |
| Use Llama-2-7B rather than the reference Bayesian model | Llama-2-7B · Bernoulli, θ=0.5 | epistemic uncertainty scaling | Hurt |
| Use GPT-3.5 rather than the reference Bayesian model | GPT-3.5 · Bernoulli, θ=0.5 | epistemic uncertainty scaling | Mixed |
| Use GPT-3-170B or Mistral-7B rather than Llama-2-7B or GPT-3.5 | LLMs on Bernoulli data · Bernoulli, θ=0.5 | epistemic uncertainty scaling | Helped |
| Fine-tune GPT-3-2.7B on Bernoulli or synthetic NLP data | GPT-3-2.7B · Bernoulli and synthetic natural language | martingale property adherence | Mixed |
| Add instruction tuning | GPT-3-170B compared with GPT-3.5, and GPT-4 · Bernoulli and Gaussian experiments | martingale property adherence | Hurt |
| Use GPT-3.5 on Gaussian data | GPT-3.5 · Gaussian, θ=-1 | sample variance: 0.711<1 (95% CI: [0.680, 0.742]) | Hurt |
Surprising findings
- Instruction-tuned GPT-3.5 and GPT-4 performed worse than base GPT-3 170B, suggesting instruction tuning may worsen non-Bayesian ICL behaviour.
- Fine-tuning GPT-3 2.7B on Bernoulli/NLP data improved martingale adherence but did not always pass the checks.
Limitations
- Restricted to small-scale synthetic datasets, different from common NLP applications.
- Checks only on short finite sampling horizons; long horizons may reflect generic approximation error.
- GPT-4 used only in a subset of experiments due to API cost and limits.
Experimental setup
Datasets: synthetic Bernoulli sequences synthetic Gaussian sequences synthetic clinical natural-language data
Benchmarks: synthetic Bernoulli experiment synthetic Gaussian experiment synthetic natural language experiment
Metrics: test statistics T_{1,g} and T_{2,k} vs bootstrap CIs IQR of MLE on sample paths (T_3) scaling
Baselines: Bayesian reference model (Beta(1,1) / Gaussian prior) fractional Bayesian posterior
Builds on: martingale posterior de Finetti's theorem Bernstein-von Mises theorem fractional Bayesian posterior
Closest papers by meaning
- In-Context Learning through the Bayesian Prism (ICLR 2024 · similarity 0.90)
- Variational Uncertainty Decomposition for In-Context Learning (NeurIPS 2025 · similarity 0.89)
- Competition Dynamics Shape Algorithmic Phases of In-Context Learning (ICLR 2025 · similarity 0.89)
- Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning (NeurIPS 2025 · similarity 0.88)
- Why Larger Language Models Do In-context Learning Differently? (ICML 2024 · similarity 0.88)
- Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.