Research map / Large language models / Language, safety and interpretability
Attributing Culture-Conditioned Generations to Pretraining Corpora
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MEMOed, a framework that attributes culture-conditioned generations from OLMo-7B to memorized culture-symbol co-occurrences in Dolma pretraining documents, across 110 cultures on food and clothing.
Problem
Open-ended culture-conditioned generations by LLMs show cultural bias and templated outputs for less prevalent cultures; it is unclear whether these outputs stem from memorization of pretraining data patterns.
Approach
Identify contributory pretraining documents for each culture-symbol pair using token distance and a document signal-to-noise ratio, then classify symbols as memorized when the contribution score is a z-score outlier across cultures; further categorize non-memorized associations (diffuse, cross-culture generalization, weak association generalization) and relate them to pretraining frequency.
Key results
- Memorized symbols account for 46.12% of unique food symbols and 25.78% of clothing symbols; food memorization correlates with culture document count (Spearman 0.670).
- Diffuse-association symbols appear at least 1000x more frequently than memorized symbols for clothing; overshadowing ratio correlates with cultures generated (Spearman 0.551).
- MEMOed memorized classification agrees with human emblematic annotations at weighted F1 0.845 (clothing) and 0.670 (food).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| more culture-related pretraining documents | culture-conditioned generations by OLMo-7B · Food and clothing, 110 cultures | number of memorized symbols: Food: Spearman 0.670; clothing: Spearman 0.540 | Helped |
| higher pretraining frequency of diffuse-association symbols | culture-conditioned generations by OLMo-7B · Food and clothing, diffuse-association symbols | number of cultures a symbol is generated for: clothing Spearman ρ = 0.551; food Spearman ρ = 0.519 | Helped |
| more culture-related pretraining documents for the culture whose symbol is memorized | cross-culture symbol generations by OLMo-7B · Food and clothing | frequency memorized symbols are generated for other cultures: clothing: Spearman ρ = 0.763; food: Spearman ρ = 0.716 | Helped |
| more culture-related pretraining documents for a culture | culture-conditioned generations by OLMo-7B · Food, cross-culture generations | percentage of responses containing another culture's memorized symbols: Spearman ρ = -0.521 | Hurt |
| lower z-score threshold from 2.6 to 2 (ablation) | MEMOed · Clothing, 8 cultures | weighted F1 against human evaluation: 0.845 -> 0.840 | Hurt |
| lower z-score threshold from 2.6 to 2 (ablation) | MEMOed · Food and clothing, culture memorization frequency relationship | Spearman correlation: food: 0.670 -> 0.688; clothing: 0.540 -> 0.569 | No clear effect |
| vary top_k to 20 or 80, or temperature to 0.75 or 1.25 (ablation) | OLMo-7B sampling for culture-conditioned generations · Food and clothing | overlap coefficient with original sampling settings: greater than 90% | No clear effect |
| use OLMo-7B-0424 instead of OLMo-7B (ablation) | culture-conditioned generations · Food and clothing, 10 selected cultures | Spearman correlation between memorized-symbol count and culture pretraining-docu: food: 0.670 -> 0.829; clothing: 0.540 -> 0.591 | No clear effect |
Surprising findings
- Many memorized symbols are not culturally emblematic (e.g., 'business suit' for Japanese clothing), which human annotators would miss.
- Weak-association generalization is nearly uncorrelated with the number of memorized symbols (Spearman 0.17 clothing, -0.03 food).
Limitations
- Only tested on OLMo-7B family because it is the most capable model with open, searchable pretraining data.
- Document-level memorization unit ignores batch ordering and multiple excerpts per document.
- Human annotation covers only 8 cultures.
Experimental setup
Datasets: Dolma
Metrics: Spearman correlation Kendall tau weighted F1 z-score overshadowing ratio
Builds on: OLMo Dolma Infinigram WIMBD
Closest papers by meaning
- CultureLLM: Incorporating Cultural Differences into Large Language Models (NeurIPS 2024 · similarity 0.88)
- Generative Monoculture in Large Language Models (ICLR 2025 · similarity 0.87)
- On Linear Representations and Pretraining Data Frequency in Language Models (ICLR 2025 · similarity 0.86)
- CulturePark: Boosting Cross-cultural Understanding in Large Language Models (NeurIPS 2024 · similarity 0.86)
- Linguistic Calibration of Long-Form Generations (ICML 2024 · similarity 0.86)
- Measuring Non-Adversarial Reproduction of Training Data in Large Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.