Research map / Large language models / Reasoning and chain of thought

Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Language-Mixed CoT (English-anchored code-switching reasoning traces) and the Yi-Sang Korean dataset (5.79M prompts, 3.7M Qwen3-32B traces, 260k HQ subset) to SFT nine 4B–35B models into Korean reasoning models (KO-REAson).

Problem

Reasoning models are mostly built for English and Chinese; mid-resource languages like Korean lack native long-CoT post-training data, and translated pipelines cause translation artifacts and degrade culture/linguistic performance.

Approach

Generate reasoning traces that code-switch between English (anchor) and Korean (target), preserving Korean named entities and key terms, from native-authored Korean prompts scraped from web Q&A, then SFT on a curated high-yield subset.

Key results

• KO-REAson-35B (A.X-3.1-35B base) achieves the highest average of 64.0 across nine benchmarks, vs. 59.6 for QwQ-32B and 58.8 for GPT-OSS-20B, ranking first on 5 of 9.
• Language-Mixed CoT (English anchor) gives Gemma-3-4B MCLM 55.8 vs 25.6 for Korean-only CoT and 48.1 for English-only CoT per the ablation table.
• Smaller and mid-sized models gain an average of +18.6 points across nine benchmarks after training on Yi-Sang-HQ.

Surprising findings

• Fine-tuning on translated OpenThoughts improved MATH but substantially degraded the Korean HAE-RAE Bench (35.24 to 15.34 for Qwen2.5-1.5B).
• Adding 500k English OpenThoughts boosted MCLM-Ko but reduced KMMLU-Redux and HAE-RAE scores, showing a trade-off toward math.
• Training on Korean-only reasoning traces without English anchoring sharply hurt math performance (MCLM 25.6 vs 48.1 English-only for Gemma-3-4B).

Limitations

• Results focus on Korean; generality to other mid-resource languages is untested beyond the case study.
• Competition-level math (AIME2024, KSM) trails GPT-OSS-20B, attributed to limited translated competition data.
• Held-in benchmarks used as iterative proxies, creating mild overfitting risk; decontamination is n-gram only.

Experimental setup

Datasets: Yi-Sang Yi-Sang-HQ OpenThoughts MCLM KMMLU-Redux HAE-RAE Bench ClinicalQA

Benchmarks: KMMLU-Redux KMMLU-Pro KMMLU-Hard MATH AIME2024 KSM HAE-RAE Bench CLIcK KoBALT-700 MCLM AIME2025 GPQA

Metrics: accuracy

Baselines: GPT-OSS-20B DeepSeek-R1-Distill-Qwen-32B EXAONE-Deep-32B QwQ-32B Gemini-2.5-Pro Monolingual English CoT Monolingual Korean CoT

Builds on: OpenThoughts Qwen3 DeepSeek-R1 S1

Closest papers by meaning

• Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.93)
• Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding (AAAI 2026 · similarity 0.92)
• Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.91)
• AdaMCoT: Rethinking Cross-Lingual Factual Reasoning Through Adaptive Multilingual Chain-of-Thought (AAAI 2026 · similarity 0.91)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.91)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.90)