Research map / Large language models / Reasoning and chain of thought

LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoongRL, a GRPO-based RL method trained on KeyChain-augmented 16K multi-hop QA (UUID key-value chains hiding the true question) that yields plan-retrieve-reason-recheck reasoning and generalizes to 128K contexts.

Problem

Long-context reasoning (not just retrieval) is underdeveloped in LLMs, and high-difficulty, verifiable RL data for long contexts is scarce; RL at 128K is also prohibitively expensive.

Approach

KeyChain synthesizes hard long-context RL data by padding real multi-hop QA with distractor documents and hiding the true question inside UUID key-value chains; training at 16K with a two-way substring exact-match verifier and a multi-stage curriculum induces a plan-retrieve-reason-recheck pattern that transfers to 128K.

Key results

• LoongRL-14B reaches 74.2 average on long-context multi-hop QA (LongBench v1), close to o3-mini (74.5) and DeepSeek-R1 (74.9), with +21.1 absolute gain over Qwen2.5-14B-Instruct.
• LoongRL-7B improves Qwen2.5-7B-Instruct by +23.5 points on long-context multi-hop QA (48.9 to 72.4 average).
• Trained at 16K, LoongRL-14B scores 79.9 on RULER at 128K versus 73.6 for Qwen2.5-14B-Instruct.

Surprising findings

• R1-distilled Qwen models degrade sharply on RULER at 128K (e.g., R1-Distill-Qwen-7B drops to 0.9) and lose performance on long-context QA at 7B.
• Replacing KeyChain data with regular multi-hop QA gives only 66.2 average versus 72.4 with KeyChain.

Limitations

• Relies on rule-based substring matching, which may be lenient and is tailored to short answer QA.
• Gains on short-context math are preserved rather than improved, and R1-distilled baselines outperform on MATH.
• Evaluation largely on multi-hop QA and retrieval; generalization to other long-context task types is less established.

Experimental setup

Datasets: HotpotQA MuSiQue 2WikiMultiHopQA DAPO-Math MATH PG19 RULER

Benchmarks: LongBench v1 LongBench v2 MMLU MATH-500 IFEval RULER Needle in a Haystack

Metrics: accuracy pass@1

Baselines: o3-mini DeepSeek-R1 GPT-4o QwQ-32B R1-Distill-Qwen-7B R1-Distill-Qwen-14B R1-Distill-Qwen-32B QwenLong-L1-32B Qwen2.5-7B-Instruct Qwen2.5-14B-Instruct

Builds on: GRPO QwenLong-L1 RULER

Closest papers by meaning

• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.92)
• Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.91)
• RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.91)
• LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards (ICLR 2026 · similarity 0.91)
• Tina: Tiny Reasoning Models via LoRA (ICLR 2026 · similarity 0.91)
• Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.91)