Research map / Large language models / Retrieval and knowledge
Co-occurrence is not Factual Association in Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that finetuned LMs learn word co-occurrence rather than factual associations from narrative text, and proposes implicit-association (Referencing) training data and active forgetting via layer-wise parameter reset to improve generalizable factual learning.
Problem
Pretrained language models struggle to learn novel factual knowledge from limited finetuning text in a form that generalizes to reasoning tasks, because they learn word co-occurrence statistics rather than true factual associations.
Approach
Co-occurrence statistics are stored in middle layers and fail to support reasoning, whereas factual associations are stored in lower layers; training on text with implicit (referencing) entity associations forces factual learning, and periodically resetting upper-layer parameters during narrative training actively forgets co-occurrence and unblocks factual learning.
Key results
- On LLaMA 3 8B, Referencing training raised multiple-choice accuracy from 58.2 (Narrative) to 98.8 and 2-hop reasoning from 38.8 to 92.5 on Country-city-animals.
- On MQuAKE-T multi-hop QA with LLaMA 3 8B, Referencing training reached 74.6 versus 61.3 for original narrative training.
- Active forgetting on upper 2/3 layers raised LLaMA 3 8B multiple-choice accuracy from 58.2 to 71.3 and 2-hop accuracy from 38.8 to 51.2 on Country-city-animals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on implicit factual associations rather than narrative text | LLaMA 3 8B · Country-city-animals | 5-shot accuracy (2-hop reasoning): 38.8 -> 92.5 | Helped |
| Train on implicit factual associations rather than narrative text | LLaMA 3 70B (LoRA) · Country-city-animals | 5-shot accuracy (Indirect reasoning): 61.0 -> 94.0 | Helped |
| Train on implicit factual associations rather than original narrative facts | LLaMA 3 8B · MQuAKE-T | 5-shot accuracy (Multi-hop QA): 61.3 (0.6) -> 74.6 (0.5) | Helped |
| Train on implicit factual associations rather than original narrative facts | LLaMA 3 8B · MQuAKE-T | 5-shot accuracy (Single-hop QA): 98.5 (0.3) -> 97.8 (0.5) | Hurt |
| Reset upper >10 layers during training | LLaMA 3 8B · Country-city-animals | Accuracy (MC): 58.2 -> 71.3 | Helped |
| Reset upper >10 layers during training | LLaMA 3 8B · 2WikiMultiHopQA | Accuracy (Multi-hop): 30.9 (0.7) -> 33.0 (0.7) | Helped |
| Reset upper >26 layers during training | LLaMA 3 70B (LoRA) · Country-city-animals | Accuracy (MC): 42.5 -> 67.3 | Helped |
| Tune only the lower <10 layers instead of the full model (ablation) | LLaMA 3 8B · Country-city-animals | Accuracy (MC): 58.2 -> 51.2 | Hurt |
| Tune only the lower <10 layers instead of the full model (ablation) | LLaMA 3 8B · MQuAKE-T | Accuracy (2-hop): 61.3 (0.6) -> 59.6 (0.8) | Hurt |
| Tune only the lower <26 layers instead of the full model (ablation) | LLaMA 3 70B (LoRA) · 2WikiMultiHopQA | Accuracy (Multi-hop): 37.3 (0.6) -> 34.4 (0.8) | Hurt |
Surprising findings
- Narrative-trained models reach near-perfect QA accuracy yet perform poorly on reasoning tasks, with the negation ratio near 1, indicating co-occurrence rather than factual learning.
- Finetuning only the lower layers does not improve generalization, because co-occurrence statistics can be learned in the lower layers too.
Limitations
- Only two forms of text (narrative and referencing) were studied; other ways of expressing facts are unexplored.
- Converting general text into implicit-association text may require complex LLM-based rewriting, which is out of scope.
- Only finetuning of pretrained models was studied, not pretraining from scratch.
Experimental setup
Datasets: Country-city-animals MQuAKE-T 2WikiMultiHopQA MQuAKE
Benchmarks: Country-city-animals reasoning tasks (QA, multiple choice, reverse QA, indirect reasoning, 2-hop reasoning) MQuAKE-T 2WikiMultiHopQA
Metrics: 5-shot exact match accuracy comparison ratio negation ratio
Baselines: plain finetuning finetuning only lower layers Narrative training data
Builds on: LoRA MQuAKE 2WikiMultiHopQA
Closest papers by meaning
- Enhancing Elusive Clues in Knowledge Learning by Contrasting Attention of Language Models (AAAI 2025 · similarity 0.88)
- From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When (NeurIPS 2024 · similarity 0.87)
- Scalable Influence and Fact Tracing for Large Language Model Pretraining (ICLR 2025 · similarity 0.87)
- Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data (ICLR 2025 · similarity 0.87)
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML 2024 · similarity 0.87)
- Emergence of Linear Truth Encodings in Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.