Research map / Large language models / Language, safety and interpretability
Understanding In-Context Learning from Repetitions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies in-context learning through surface repetitions, establishing a token co-occurrence reinforcement effect in LLMs and showing it both helps (output-space constraint, format following) and harms (spurious connections) ICL.
Problem
The working mechanism of in-context learning in LLMs is unclear, and existing explanations do not account for how surface textual features and repetition patterns in demonstrations drive ICL behavior and its failures.
Approach
Frames ICL through self-reinforcement of surface repetitions, showing that the probability of generating a token rises with the number of its contextual co-occurrences with another token (token co-occurrence reinforcement), even for random tokens, and that this mechanism both enables and undermines ICL.
Key results
- On LLaMA-65B, accuracy on MMLU class D rises from 52.28 to about 77.19 when 100% of demonstration answers are reordered to D (5 demos), while classes A-C decline to 58.05 average.
- Non-informative phrase before answer D raises class-D accuracy by 14.15 points with one demo while reducing average A/B/C accuracy by 1.83 points.
- Repeating sentences with random tokens drives token probability toward near 1.0 across OPT and LLaMA models after 10 repeats.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a non-informative phrase before Answer: in D-answer demonstrations | LLaMA-65B · MMLU | D accuracy: 59.53 -> 61.64 | Helped |
| Add a non-informative phrase before Answer: in D-answer demonstrations | LLaMA-65B · MMLU | Avg. [A,B,C] accuracy: 64.37 -> 63.47 | Hurt |
| Replace all demonstrated answers with D and swap the corresponding option contents | LLaMA-65B · MMLU | D accuracy: 59.53 -> 77.19 | Helped |
| Replace all demonstrated answers with D and swap the corresponding option contents | LLaMA-65B · MMLU | Avg. [A,B,C] accuracy: 64.37 -> 58.05 | Hurt |
| Replace 75% of demonstrated answers with D and swap the corresponding option contents | LLaMA-65B · MMLU | D accuracy: 59.53 -> 66.90 | Helped |
| Replace 75% of demonstrated answers with D and swap the corresponding option contents | LLaMA-65B · MMLU | Avg. [A,B,C] accuracy: 64.37 -> 62.34 | Hurt |
| Mask question and option contents in demonstrations while retaining formatting words and final answers (ablation) | LLaMA-65B · MMLU | probability of choosing a label in ABCD | No clear effect |
| Replace option names and answer indicators with semantically equivalent substitutes (ablation) | LLaMA-65B · MMLU | probability of choosing a label in ABCD | Hurt |
| Mask the repeated // separator in chain-of-thought demonstrations (ablation) | LLaMA · GSM8K | probability of generating the CoT pattern | Hurt |
| Mask the question content in chain-of-thought demonstrations (ablation) | LLaMA · GSM8K | probability of generating the CoT pattern | Hurt |
| Prevent each token from attending to its preceding token (ablation) | LLaMA-65B · Successive token reinforcement | reinforcement strength: a factor of ten | Hurt |
Surprising findings
- Even sentences of random tokens, which start with near-zero probability, become reinforced to near-1 probability through repetition.
- Adding a non-informative phrase before the answer indicator shifts LLaMA-65B predictions toward choice D, overriding its selection bias.
Limitations
- Only studies repeating surface patterns; more sophisticated patterns are not examined.
- Mechanistic explanation via attention is limited to a single ablation on LLaMA-65B.
- Analyses rely on a small set of models and do not give a complete account of ICL.
Experimental setup
Datasets: Wikitext-103 BookCorpus GSM8K MMLU wikipedia-english-2022
Benchmarks: MMLU GSM8K
Metrics: token probability accuracy probability of label space probability of CoT pattern improved ratio
Baselines: zero-shot prompting standard ICL with balanced demonstrations
Builds on: self-reinforcement effect chain-of-thought prompting induction heads
Closest papers by meaning
- Dual Operating Modes of In-Context Learning (ICML 2024 · similarity 0.88)
- From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When (NeurIPS 2024 · similarity 0.88)
- Where does In-context Learning Happen in Large Language Models? (NeurIPS 2024 · similarity 0.88)
- Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting (ICLR 2025 · similarity 0.88)
- Toward Understanding In-context vs. In-weight Learning (ICLR 2025 · similarity 0.87)
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.