Research map / Large language models / Alignment and preferences
SafeWorld: Geo-Diverse Safety Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SafeWorld, a 2,342-query benchmark for geo-diverse (cultural and legal) safety, with a multi-dimensional evaluation, and SafeWorldAlign, a DPO dataset of 45,746 preference pairs used to train SafeWorldLM on Zephyr-7B-SFT.
Problem
LLM safety evaluation and alignment overlook geo-diversity of cultural norms and legal policies, so models may give responses that are inappropriate or inaccurate across countries and regions.
Approach
Build a bottom-up, human-verified database of country- and region-level cultural norms and policies, generate four query types tied to these guidelines, score responses on response-type match, reference faithfulness/coverage, and reference-free factuality, and align a 7B model with DPO using two kinds of negative responses (wrong response type and wrong guideline).
Key results
- SafeWorldLM (7B) reaches 0.521 average coverage and 0.473 response type matching, versus 0.301 coverage and 0.268 type matching for GPT-4o.
- Human global annotators from 9 countries give SafeWorldLM an 18-20% higher winning rate than GPT-4o on helpfulness and harmlessness.
- SafeWorldLM keeps MMLU (56.6 vs 56.8) and HellaSwag (78.5) near the Zephyr-7B-SFT-Full base, while raising HH-RLHF harmless ratio from 59.3% to 96.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DPO alignment using SafeWorldAlign preference pairs | Zephyr-7B-SFT-Full · SafeWorld | coverage: 0.301 -> 0.521 | Helped |
| Apply DPO alignment using SafeWorldAlign preference pairs | Zephyr-7B-SFT-Full · SafeWorld | faithfulness: 0.141 -> 0.149 | Helped |
| Apply DPO alignment using SafeWorldAlign preference pairs | Zephyr-7B-SFT-Full · SafeWorld | factuality: +0.068 | Helped |
| Apply DPO alignment using SafeWorldAlign preference pairs | Zephyr-7B-SFT-Full · SafeWorld | response type match: +20.5% | Helped |
| Remove Negative Category 1 pairs (ablation) | SafeWorldLM · SafeWorld | factuality: 0.628 -> 0.676 | Helped |
| Remove Negative Category 1 pairs (ablation) | SafeWorldLM · SafeWorld | coverage: 0.521 -> 0.366 | Hurt |
| Remove Negative Category 1 pairs (ablation) | SafeWorldLM · SafeWorld | faithfulness: 0.149 -> 0.096 | Hurt |
| Remove Negative Category 1 pairs (ablation) | SafeWorldLM · SafeWorld | response type match: 0.473 -> 0.392 | Hurt |
| Remove Negative Category 2 pairs (ablation) | SafeWorldLM · SafeWorld | response type match: 0.473 -> 0.516 | Helped |
| Remove Negative Category 2 pairs (ablation) | SafeWorldLM · SafeWorld | coverage: 0.521 -> 0.469 | Hurt |
| Remove Negative Category 2 pairs (ablation) | SafeWorldLM · SafeWorld | faithfulness: 0.149 -> 0.164 | Helped |
| Remove Negative Category 2 pairs (ablation) | SafeWorldLM · SafeWorld | factuality: 0.628 -> 0.564 | Hurt |
Surprising findings
- GPT-4-turbo struggles with queries implicitly tied to guidelines it generated itself, and is worse at response-type matching than some open-source models.
- Open-source LLaMA and Mistral models match or exceed GPT-3.5-turbo and Command-R/R+ on several dimensions, and some exceed GPT-4-turbo on response-type classification.
Limitations
- Cultural-legal guidelines are generated by GPT-4-turbo, and policy validation relied on machine verification without human annotators due to budget.
- Country coverage skews toward non-Western countries and cultural norm coverage is narrower than policy coverage; some regions lacked qualified annotators.
- Evaluation depends on GPT-4-turbo as judge, with Llama-3-70B-Instruct showing only moderate correlation with humans on coverage and faithfulness.
Experimental setup
Datasets: SafeWorldAlign UltraFeedback Safer-Instruct MMLU HellaSwag Anthropic HH-RLHF BeaverTails
Benchmarks: SafeWorld MMLU HellaSwag Anthropic HH-RLHF BeaverTails
Metrics: response type matching faithfulness coverage factuality win rate harmless response ratio accuracy
Baselines: Zephyr-7B-SFT-Full Llama-2-7B-chat Llama-2-13B-chat Llama-3-8B-Instruct Mistral-7B-Instruct-v0.1 Mistral-7B-Instruct-v0.2 Command-R Command-R+ GPT-3.5-turbo GPT-4-turbo GPT-4o
Builds on: DPO Zephyr-7B-SFT-Full Alignment Handbook INSTRUCTOR K-Means
Closest papers by meaning
- Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.90)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.89)
- CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models (ICML 2025 · similarity 0.88)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.88)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.88)
- RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios? (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.