Research map / Large language models / Language, safety and interpretability
CALM: Culturally Self-Aware Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CALM, a Qwen3-32B-based framework that disentangles task, explicit cultural and latent cultural streams, aligns them via cross-attention and dimension-specific MoE, and adds a self-prompted reflective loop, reporting gains on cultural commonsense, value, hate speech and bias benchmarks.
Problem
Existing cultural awareness approaches in language models treat culture as static background knowledge or external prompts, limiting reliability on tasks requiring cultural sensitivity.
Approach
Model culture as a dynamic internal identity state: disentangle explicit cultural concepts and latent cultural signals, cluster them with contrastive learning and Gumbel-Softmax, align with cross-attention, route through expert-choice MoE over three communicative dimensions, and self-correct via reverse identity calibration.
Key results
- On CultureAtlas, CALM (Qwen3-32B) reaches 89.1% F1 overall versus 93.1% for GPT-4, with 92.4% F1 in high-resource groups.
- On CREHate, CALM averages 83.66% accuracy versus 78.20% for GPT-4.
- On EMGSD, CALM reaches 85.3% Macro F1 versus 82.8% for BERT and 64.8% for GPT-4o.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CALM instead of GPT-4 | CultureAtlas (All Culture) · CultureAtlas (All Culture) | F1: 89.1 vs 93.1 | Hurt |
| Use CALM instead of GPT-4 | CREHate · CREHate (Avg) | accuracy: 78.20 -> 83.66 | Helped |
| Use CALM instead of BERT | EMGSD · EMGSD | Macro F1: 82.8 -> 85.3 | Helped |
| add the latent cultural signal (ablation) | CALM · CultureAtlas (All Culture) | F1: 87.8 -> 89.1; 1.3% reduction when removed | Helped |
| add the explicit cultural concept (ablation) | CALM · CultureAtlas (All Culture) | F1: 87.1 -> 89.1; 2.0% reduction when removed | Helped |
| add the contrastive window (ablation) | CALM · CultureAtlas (All Culture) | F1: 2.2% drop when removed | Helped |
| add the identity alignment pool (ablation) | CALM · CultureAtlas (All Culture) | F1: 9.0% decrease when removed | Helped |
| use cross-attention instead of shallow fusion (ablation) | CALM · CultureAtlas (All Culture) | F1: 2.9% drop with shallow MLP replacement | Helped |
| add cross-attention between cultural streams (ablation) | CALM · CultureAtlas (All Culture) | F1: 4.6% F1 drop when integration is removed | Helped |
| use MoE instead of a shared MLP (ablation) | CALM · CultureAtlas (All Culture) | F1: 5.6% F1 drop with shared MLP | Helped |
| organize experts by cultural dimensions (ablation) | CALM · CultureAtlas (All Culture) | F1 | Helped |
| use dynamic expert routing instead of uniform averaging (ablation) | CALM · CultureAtlas (All Culture) | F1: 6.0% F1 drop with uniform averaging | Helped |
Surprising findings
- CALM's reported F1 on CultureAtlas (89.1%) is below GPT-4's 93.1%, despite the paper stating comparable performance.
- Ablation with other backbones shows CALM's gains are attributed to architecture rather than scale, per the authors.
Limitations
- Baselines are official benchmark baselines and compute-heavy 32B backbone; efficiency limited (VRAM >67 GB).
Experimental setup
Datasets: CultureAtlas UniVaR CREHate EMGSD SBIC MGSD WinoQueer SeeGULL
Benchmarks: CultureAtlas UniVaR CREHate EMGSD
Metrics: precision recall F1 accuracy Macro F1 Acc@1 Acc@5 Acc@10 k-NN accuracy
Baselines: GPT-4 GPT-4o GPT-3.5 LLaMA-2-7B Vicuna-13B Orca-2 Flan-T5 BERT DistilBERT ALBERT-V2
Builds on: SimCLR Gumbel-Softmax Mixture-of-Experts Hall's high/low-context theory Brown and Levinson politeness theory
Closest papers by meaning
- Culture In a Frame: C$^3$B as a Comic-Based Benchmark for Multimodal Culturally Awareness (ICLR 2026 · similarity 0.86)
- CultureLLM: Incorporating Cultural Differences into Large Language Models (NeurIPS 2024 · similarity 0.85)
- CulturePark: Boosting Cross-cultural Understanding in Large Language Models (NeurIPS 2024 · similarity 0.85)
- A Theoretical Understanding of Self-Correction through In-context Alignment (NeurIPS 2024 · similarity 0.84)
- Attributing Culture-Conditioned Generations to Pretraining Corpora (ICLR 2025 · similarity 0.84)
- Communication Accommodation Between Large Language Models and Users Across Cultures (Student Abstract) (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.