Research map / Large language models / Reasoning and chain of thought

Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematic comparison of RL (GRPO) vs SFT for cross-lingual reasoning on Qwen2.5-3B-Base, finding RL generalizes better across languages and that non-English (German) RL training beats English RL, with analyses of language inconsistency, sampling, and representation shift.

Problem

Whether RL generalizes across languages better than SFT for multilingual reasoning, and whether training on non-English data is better than English data for RL.

Approach

Compares RL (GRPO) and SFT trained on translated GSM8K/LUFFY data in English, Chinese, German, and other languages, showing RL gives stronger cross-lingual transfer and that non-English RL training outperforms English RL, explained via language inconsistency in reasoning, on-policy sampling (vs RFT), and smaller hidden-state drift.

Key results

• On MGSM with Qwen2.5-3B-Base, RL (De) reaches 71.5% average accuracy vs 46.3% for SFT (De), with a generalization score of 60.4 vs 19.3.
• RL trained on German achieves 71.5% average on MGSM versus 62.7% for RL trained on English.
• On MMath500, RL (Zh) averages 61.3% vs SFT (Zh) 39.8%; SmolLM3-3B-Base shows the same trends with RL (De) averaging 69.9 vs SFT (De) 41.8 on MGSM.

Surprising findings

• Non-English (German, Chinese) RL training outperforms English RL training, while SFT shows no such advantage.
• Enforcing language consistency in reasoning degrades RL performance (e.g., RL (De) falls from 61.4% to 52.0% on MMath500), suggesting language inconsistency aids cross-lingual reasoning.

Limitations

• Experiments limited to small base models (3B) and a few languages; generalization to larger models is not shown.
• Mechanistic explanations (language inconsistency, sampling, representation shift) are preliminary and largely correlational.
• Training data are machine translations by Qwen3-30B-A3B, with only verification-based quality control.

Experimental setup

Datasets: GSM8K (translated) LUFFY (translated)

Benchmarks: MGSM MMath500 MAIME2024 MMLU-ProX-Lite MGPQA-D

Metrics: accuracy pass@k generalization score (Gen)

Baselines: SFT RFT Base model

Builds on: GRPO GSM8K MGSM LUFFY Rejection Sampling Fine-Tuning

Closest papers by meaning

• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.93)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.93)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.92)
• Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.92)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.91)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.91)