Research map / Large language models / Instruction and fine-tuning

Soft Token Alignment for Cross-Lingual Reasoning

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SOLAR, an auxiliary SFT loss that aligns mean-pooled soft-token embeddings of non-English reasoning traces to English counterparts, improving multilingual reasoning accuracy and cross-lingual consistency.

Problem

Multilingual LLMs give inconsistent reasoning and answers for semantically equivalent prompts across languages, with final-layer representations becoming language-specific as discrete tokens are selected.

Approach

Uses soft tokens (probability-weighted mixtures of vocabulary embeddings) computed from the teacher-forced forward pass as a language-agnostic representation; mean-pools them per response and applies a cosine alignment loss pulling non-English summaries toward the English pivot during SFT, without changing inference.

Key results

• SOLAR on Qwen3-4B reaches 48.25% average accuracy vs. 45.67% for SFT (+2.6 avg) and 38.92% base; Swahili MGSM rises from 29.04% to 53.48%.
• On Qwen3-8B, SOLAR achieves 70.21% average accuracy vs. 68.65% for SFT, with CLC 64.55 vs. 62.31.
• Final-layer CLS for Qwen3-4B rises from 0.30 (SFT) to 0.50 (SOLAR) and silhouette drops from 0.86 to 0.77.

Surprising findings

• Discrete-token alignment (k=1) recovers almost none of SOLAR's gain, indicating top-k soft-token truncation carries semantic value.
• Inference-time soft thinking does not consistently improve cross-lingual reasoning and slightly reduces SOLAR's final-layer CLS when applied on top of it.

Limitations

• Requires parallel multilingual reasoning data (M-s1k covers only 7 evaluated languages); expansion costly.
• Evaluated only on math and science short-answer tasks; open-ended generation untested.
• Main SFT experiments limited to Qwen3 and Gemma3 families at 4B-12B scale.

Experimental setup

Datasets: M-s1k s1k MGSM AIME 2024 AIME 2025 GPQA Diamond

Benchmarks: MGSM AIME 2024 AIME 2025 GPQA Diamond

Metrics: accuracy cross-lingual consistency (CLC) cross-lingual similarity (CLS) silhouette score target-script ratio

Baselines: Soft Thinking SFT MidAlign MAPO AlignX

Builds on: soft thinking chain-of-thought MidAlign MAPO AlignX

Closest papers by meaning

• Multilingual LLMs Inherently Reward In-Language Time-Sensitive Semantic Alignment for Low-Resource Languages (AAAI 2025 · similarity 0.90)
• AdaMCoT: Rethinking Cross-Lingual Factual Reasoning Through Adaptive Multilingual Chain-of-Thought (AAAI 2026 · similarity 0.89)
• Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought (ICLR 2026 · similarity 0.89)
• XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning (AAAI 2025 · similarity 0.89)
• When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (NeurIPS 2025 · similarity 0.89)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.88)