Research map / Large language models / Reasoning and chain of thought

RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RLAD, a two-player RL method that jointly trains an abstraction generator and an abstraction-conditioned solution generator so that models learn to propose and use natural-language reasoning abstractions for math problems.

Problem

RL post-training on long chains of thought tends to favor depth over breadth, producing verbose, degenerate exploration rather than reusable procedures; the paper asks how models can explore a diversity of reasoning strategies on hard problems.

Approach

Train an LLM to propose concise natural-language reasoning abstractions per problem, and jointly train a solution generator with RL to exploit them; the abstraction generator is rewarded by the solution generator's accuracy conditioned on its abstraction, with zeroed reward for no-abstraction prompts to prevent ignoring abstractions.</near>

Key results

• On AIME 2025 with Qwen3-1.7B, RLAD w/ abs (avg) reaches 42.45 vs 37.92 for DAPO w/o abs (avg 38.04 without abstraction).
• On DeepScaleR Hard, RLAD w/ abs (best) achieves 35.54 vs 21.67 for the base Qwen3-1.7B w/o abs.
• Under equal compute on AIME 2025, n abstractions x n solutions yields pass@k 0.71 vs 0.65 for n^2 plain samples at n=16.

Surprising findings

• Training with abstractions improves performance even in the no-abstraction setting (e.g., AIME 2024 51.46 vs 48.54 base).
• A single model trained to both propose and use abstractions quickly lost its ability to propose abstractions during RL.

Limitations

• Evaluation limited to mathematical reasoning; open-ended reasoning unexplored.
• Abstraction generator and solution generator are separate models; a single unified model lost abstraction proposal ability.
• Warmstart abstractions depend on o4-mini summarization and GPT-4.1-mini filtering, and RL used offline batched updates due to infrastructure limits.

Experimental setup

Datasets: DeepScaleR AIME 2025 AMC 2023 BARC RAFT CLUES LegalBench

Benchmarks: AIME 2025 DeepScaleR Hard AMC 2023 AIME 2024 HMMT 2025 ARC-AGI (90-puzzle subset)

Metrics: pass@1 accuracy pass@k max@k coverage adherence rate semantic similarity

Baselines: DAPO Qwen3-1.7B base

Builds on: DAPO GRPO Qwen3 RFT DeepScaleR

Closest papers by meaning

• AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking (ICLR 2026 · similarity 0.90)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.89)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.89)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.88)
• RLKD: Distilling LLMs’ Reasoning via Reinforcement Learning (AAAI 2026 · similarity 0.88)