Research map / Large language models / Language, safety and interpretability
Pre-trained Large Language Models Learn to Predict Hidden Markov Models In-context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows pre-trained LLMs (Qwen2.5, Llama) can predict synthetic HMM sequences via in-context learning, approaching Viterbi-oracle accuracy, with scaling trends tied to entropy and mixing rate, and beats GLM-HMM on IBL mouse decision data.
Problem
Fitting Hidden Markov Models to real-world sequential data is computationally hard and unstable (Baum-Welch non-convexity), and it is unknown whether pre-trained LLMs can implicitly learn HMM dynamics in context.
Approach
Treat HMM-generated observation sequences as text prompts and evaluate pre-trained LLMs' next-observation prediction as the context grows, comparing against Viterbi/oracle, Baum-Welch, spectral learning, n-grams, LSTM and Transformer; relate scaling trends to HMM entropy and mixing rate with a spectral-learning-based theoretical conjecture.
Key results
- On synthetic HMMs, LLM ICL accuracy converges to the Viterbi oracle under low entropy and fast mixing, outperforming Baum-Welch, LSTM and Transformer baselines.
- On IBL mouse data, LLM ICL achieves an average 86.2% accuracy with contexts over 1000 trials, exceeding the GLM-HMM's reported 82.2%.
- On the reward-learning rat task, LLM ICL improves only marginally with context and performs substantially below the state-of-the-art evolutionary symbolic-program model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM ICL instead of GLM-HMM | IBL mouse decision-making · IBL mouse decision-making | prediction accuracy: 82.2% -> 86.2% | Helped |
| Increase context length | LLM ICL · Synthetic HMM sequence prediction | prediction accuracy | Helped |
| Use higher transition or emission entropy | LLM ICL on synthetic HMMs · Synthetic HMM sequence prediction | convergence steps | Hurt |
| Use slower mixing (higher λ₂) | LLM ICL on synthetic HMMs · Synthetic HMM sequence prediction | convergence steps | Hurt |
| Vary the number of states while holding entropy constant | LLM ICL on synthetic HMMs · Synthetic HMM sequence prediction | convergence rate | No clear effect |
| Use LLM ICL instead of empirical learning baselines | Synthetic HMM prediction · Synthetic HMM sequence prediction | prediction accuracy | Helped |
| Use a Transformer instead of an LSTM | Trained neural sequence baselines · Synthetic HMM sequence prediction | prediction accuracy | Helped |
| Use a bigram instead of other n-gram baselines | n-gram prediction baselines · Synthetic HMM sequence prediction | prediction accuracy | Helped |
| Increase context length | LLM ICL · Reward-learning rats | prediction accuracy | Helped |
| Use LLM ICL instead of the specialized symbolic-program model | Reward-learning rats · Reward-learning rats | prediction accuracy | Hurt |
| Use a truncated-history conditional predictor in a fast-mixing regime | Oracle conditional predictor · Synthetic HMM sequence prediction | prediction accuracy | Helped |
Surprising findings
- LLM in-context prediction often surpasses Baum-Welch and approaches the true-parameter Viterbi oracle, despite never being trained on HMM data.
- Number of hidden states, when entropy is held constant, does not measurably affect LLM convergence rate.
Experimental setup
Datasets: Synthetic HMM sequences IBL mice decision-making dataset Reward-learning rats dataset
Benchmarks: synthetic HMM prediction IBL mice decision-making reward-learning rats
Metrics: prediction accuracy Hellinger distance convergence step T accuracy gap to Viterbi
Baselines: Viterbi (oracle) P(O_{t+1}|O_{t-k:t}) oracle Baum-Welch spectral learning n-gram LSTM Transformer GLM-HMM
Builds on: Viterbi algorithm Baum-Welch spectral learning of HMMs GLM-HMM
Closest papers by meaning
- The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains (NeurIPS 2024 · similarity 0.89)
- In-Context Language Learning: Architectures and Algorithms (ICML 2024 · similarity 0.89)
- Competition Dynamics Shape Algorithmic Phases of In-Context Learning (ICLR 2025 · similarity 0.88)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.87)
- Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective (ICML 2024 · similarity 0.87)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.