Research map / Large language models / Alignment and preferences
Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes "native alignment," rewriting pre-training data with alignment workers during continued pre-training of Llama-3 to build Arabic LLMs (LLaMA3-Tamed-8B/70B) that are safer and more helpful.
Problem
Alignment of LLMs is usually applied after pre-training (instruction tuning/RLHF), which is costly and unstable; the paper asks whether aligning pre-training data itself (for Arabic) can improve alignment and performance.
Approach
Rewrite a portion of raw pre-training data with a code of conduct (format, fair values, content moderation, knowledge preservation) using alignment workers distilled from GPT-4 annotations, then continue pre-training on the rewritten data alongside original data ("native alignment").
Key results
- LLaMA3-Tamed-70B reaches average 69.88 across Arabic benchmarks vs 68.96 for Llama3-70B in few-shot evaluation, with AraTrust rising from 60.54 to 63.41.
- Native alignment rewriting reduces OpenAI Moderation toxicity scores by 16.5% to 31.8% across harassment, hate, sexual and violence categories on 8k Arabic samples.
- On BeaverTails (GPT-4 judged), native alignment improves harmlessness and helpfulness by 10.4% and 4.8% respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| rewrite Arabic pre-training data using native alignment | Arabic data (8k) · Toxicity detection: harassment | OpenAI Moderation score: 0.0293 -> 0.0232 | Helped |
| rewrite Arabic pre-training data using native alignment | Arabic data (8k) · Toxicity detection: hate | OpenAI Moderation score: 0.0067 -> 0.0049 | Helped |
| rewrite Arabic pre-training data using native alignment | Arabic data (8k) · Toxicity detection: sexual | OpenAI Moderation score: 0.0022 -> 0.0015 | Helped |
| rewrite Arabic pre-training data using native alignment | Arabic data (8k) · Toxicity detection: violence | OpenAI Moderation score: 0.0127 -> 0.0106 | Helped |
| add native alignment during pre-training | Llama-3-8B · ArabicMMLU | score: 45.78 -> 50.17 | Helped |
| add native alignment during pre-training | Llama-3-70B · ACVA clean | score: 83.70 -> 82.58 | Hurt |
| combine original unaligned pre-training data with alignment data, rather than train on alignment data alone (ablation) | Llama-3-8B · BeaverTails | helpfulness | Helped |
| add native-alignment data to conventional pre-training (ablation) | Llama-3-8B · BeaverTails | harmlessness: 10.4% | Helped |
| add native-alignment data to conventional pre-training (ablation) | Llama-3-8B · BeaverTails | helpfulness: 4.8% | Helped |
| increase the volume of native-alignment data (ablation) | Llama-3-8B · BeaverTails | helpfulness | Helped |
| increase the volume of native-alignment data (ablation) | Llama-3-8B · BeaverTails | harmlessness | Helped |
| use native-aligned data instead of RefinedWeb data cleaning alone (ablation) | Qwen-1.5-0.5B · MMLU Avg. | score: 27.71 -> 30.73 | Helped |
Surprising findings
- Native alignment combined with DPO gave lower average on Arabic benchmarks than native alignment with SFT alone (53.21 vs 54.12), suggesting DPO does not add gains in that setting.
- Harmlessness improved only gradually with more alignment data while helpfulness rose sharply, and Llama-3 already scored highest on BeaverTails before alignment.
Limitations
- No suitable fair benchmark for alignment, so native alignment is not directly compared to post-alignment methods; the DPO comparison is acknowledged as not apples-to-apples.
- Hallucinations inherited from original data into rewritten data; only manual review of 90 pairs.
- Instruction-following capability untested since it depends more on instruction data than pre-training data.
Experimental setup
Datasets: ArabicText2022 SlimPajama MAP-CC Wikipedia Proof-Pile-2 Alpaca-Arabic-GPT4 BeaverTails RefinedWeb ArabicMMLU EXAMS ACVA AraTrust
Benchmarks: ArabicMMLU EXAMS ACVA AraTrust BeaverTails MMLU
Metrics: accuracy harmlessness win-rate helpfulness win-rate perplexity OpenAI Moderation scores toxicity scores
Baselines: Llama3-8B Qwen1.5-7B Jais-30B-v3 Qwen1.5-32B Qwen1.5-72B Llama3-70B ChatGPT 3.5 Turbo GPT-4 AceGPT Jais
Builds on: Llama-3 Qwen1.5 superficial alignment hypothesis DPO Aligner
Closest papers by meaning
- ALLaM: Large Language Models for Arabic and English (ICLR 2025 · similarity 0.90)
- Aligner: Efficient Alignment by Learning to Correct (NeurIPS 2024 · similarity 0.89)
- The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024 · similarity 0.89)
- Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models (NeurIPS 2025 · similarity 0.89)
- Aligning Large Language Models with Representation Editing: A Control Perspective (NeurIPS 2024 · similarity 0.89)
- Exploring Intrinsic Alignments Within Text Corpus (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.