Research map / Large language models / Reasoning and chain of thought

PRISM: Demystifying Retention and Interaction in Mid-Training

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Empirical study of mid-training (math/code/science data, ~27B tokens) across seven open base models (3B-24B, dense and hybrid), showing it boosts reasoning, preserves general ability, and makes subsequent RL far more effective, with mechanistic weight and CKA analyses.

Problem

Mid-training for LLM reasoning is used inconsistently and its design choices, retention of general abilities, long-context performance, and interaction with RL are poorly understood.

Approach

A controlled, retention-aware empirical framework that varies mid-training data mixtures, timing, context length, and token budget, then chains RL on top, evaluating across families and architectures and analyzing weight and representation changes from mid-training versus RL.

Key results

• Mid-training with PRISM raises the six-benchmark AVG from 7.49 to 21.33 (LLaMA-3.1), 11.19 to 29.48 (Granite-3.3), and 9.20 to 25.05 (Mistral-7B), and full mid-train plus RL reaches 29.69, 41.76, and 35.14 respectively.
• Mistral-7B MATH500 improves from 1.68 to 70.71 after mid-training; Mistral-Small 24B improves MATH500 from 26.92 to 80.80.
• Mid-training changes over 90% of parameters with L2 divergence 370-580x larger than RL, which changes about 5% of parameters; RL preserves mid-training CKA above 0.998.

Surprising findings

• RL applied directly to base models yields near-zero AIME scores for most models, while RL on mid-trained checkpoints succeeds, even though RL weight footprints are nearly identical with or without mid-training.
• Changing the RL data mix changes AVG by under 2 points, while changing the mid-training mix shifts it by 3-6 points.

Limitations

• Timing analysis based on a single model (Granite-4 Micro 3B), so ordering may not generalize.
• Hybrid vs dense architecture attribution confounded by pretraining data and scale.
• Mid-training severely degrades long-context retrieval (RULER@128k drops from 59 to 6) unless restored with extra long-context training.

Experimental setup

Datasets: DCLM-EDU Open-R1 (MoT) Nemotron Post-Training v1 Megamath-Web-Pro OpenCodeReasoning-2 RefinCode StarCoder2 OpenThoughts3 WildChat-1M Tulu-3 SFT Personas UltraChat-200k DeepScaleR-Preview

Benchmarks: LiveCodeBench Codeforces AIME24 AIME25 MATH500 GPQA-Diamond RULER Open LLM Leaderboard V1 Open LLM Leaderboard V2 AIME 2026

Metrics: accuracy pass rate macro-average RULER score normalized L2 divergence linear CKA update sparsity

Baselines: Granite-3.3 base LLaMA-3.1 base Mistral-7B base Nemotron-H base RL on base models without mid-training

Builds on: OctoThinker GRPO

Closest papers by meaning

• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.93)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.92)
• Reinforcement Mid-Training (ICLR 2026 · similarity 0.91)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.91)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.90)