Research map / Large language models / Reasoning and chain of thought

LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LongRLVR, which augments outcome-only RLVR with a dense, verifiable context reward based on F-score over selected evidence chunk identifiers, to improve long-context grounding and QA.

Problem

Outcome-only RLVR fails on long-context tasks because the sparse final-answer reward gives vanishing gradients for learning to retrieve the relevant evidence chunks, causing contextual recall to stagnate.

Approach

Factorize the policy into a grounding head (selecting chunk identifiers) and an answer head, and add a modulated F-beta context reward over selected chunks so the grounding policy receives a dense, verifiable learning signal alongside the answer reward.

Key results

• Qwen2.5-14B-1M LongRLVR raises RULER-QA average from 73.17 (RLVR) to 88.90 and LongBench v2 overall from 39.8 to 46.5.
• LLaMA-3.1-8B with LongRLVR reaches 80.33 RULER-QA average vs 67.80 for RLVR.
• Qwen2.5-7B-1M LongRLVR achieves 79.22 on LongReason average, above LLaMA-3.1-70B (57.59) and Qwen2.5-72B-YaRN (72.76).

Surprising findings

• Qwen2.5-14B-1M LongRLVR outperforms Qwen3-14B (thinking mode) on RULER-QA (88.90 vs 87.60) and LongReason (78.42 vs 77.50).

Limitations

• Relies on a synthetic training dataset generated by a large LLM, with ground-truth evidence chunks assumed to be the essential set.
• Theoretical analysis assumes a simplified log-linear or independent Bernoulli grounding policy and a monotone set-function reward.
• Chunk-identifier-based grounding depends on a fixed chunking scheme (64 segments per document).

Experimental setup

Datasets: Long-Data-Collection StarCoder RULER-QA LongBench v2 LongReason

Benchmarks: RULER-QA LongBench v2 LongReason

Metrics: accuracy F-beta score

Baselines: SFT naive RLVR (GRPO) LLaMA-3.1-70B Qwen2.5-72B-YaRN Qwen3-8B Qwen3-14B QwenLong-L1-32B

Builds on: RLVR GRPO QwenLong-L1 YaRN

Closest papers by meaning

• The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.92)
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.92)
• LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts (ICLR 2026 · similarity 0.91)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.91)
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.91)
• Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.91)