Research map / Large language models / Reasoning and chain of thought

Long Chain-of-Thought Reasoning Across Languages

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically studies long chain-of-thought reasoning in nine non-English languages across scaling, pretraining, post-training, and inference, finding target-language long CoT lags English and that translated English reasoning traces are an effective synthetic data source.

Problem

Long chain-of-thought reasoning has been studied almost exclusively in English; it is unclear whether long-form reasoning transfers to other languages, especially when models must reason in the user's language.

Approach

Decomposes multilingual reasoning into input comprehension (En-CoT: target-language input, English reasoning) versus target-language reasoning (Target-CoT), and evaluates across model scale, base-model pretraining, and post-training data curation (translated vs. distilled s1k traces) plus inference analysis.

Key results

• At 32B, Target-CoT on AIME-Combined trails En-CoT; switching from English to target-language reasoning reduces accuracy by 28.8% on average at 32B, and 32B target-language reasoning falls at or below the 7B En-Only baseline.
• Specialized math pretraining (Qwen2.5-Math-7B vs Qwen2.5-7B) lowers Target-CoT on MATH-500 (e.g., French -46.0%) while Qwen3-8B-Base recovers 35-65% EPR in Target-CoT on MATH-500 and 95%+ EPR in En-CoT.
• Fine-tuning Qwen3-8B-Base on 1,000 translated s1k traces yields 78.3 average on MATH-500 across non-English languages vs 62.6 for 20k OpenThoughts3 English traces, with gains concentrated in mid- and low-resource languages.

Surprising findings

• Target-language reasoning at 32B performs at or below a 7B English-reasoning baseline despite ~4x fewer parameters.
• Specialized math pretraining degrades target-language reasoning even in Chinese, which was included in its specialized training data.

Limitations

• Target-CoT comparisons conflate language compliance with reasoning accuracy; results are filtered by language compliance checks.
• Error analysis uses 100 samples per language from one model and an LLM judge validated by native speakers on only four languages.
• Translation uses Gemini-2.0-Flash for benchmarks and training data, which may introduce translation artifacts.

Experimental setup

Datasets: s1k OpenThoughts3-1.2M OpenThoughts3-20k MATH-500 AIME 2024 AIME 2025 MCLM MMLU-ProX mC4 FLORES-200

Benchmarks: MATH-500 AIME-Combined MMLU-ProX

Metrics: pass@1 accuracy English performance recovered (EPR) language compliance rate average response tokens

Baselines: Qwen3-8B (Thinking) English-s1k OpenThoughts3-20k fine-tuning Qwen2.5-7B Gemma3-12B-PT

Builds on: s1k OpenThoughts3 DeepSeek-R1 language forcing Qwen2.5-Math

Closest papers by meaning

• Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding (AAAI 2026 · similarity 0.93)
• Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought (ICLR 2026 · similarity 0.93)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.93)
• Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.92)
• Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.92)
• Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation (NeurIPS 2026 · similarity 0.92)