Research map / Large language models / Reasoning and chain of thought

RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains Qwen3 4B/8B base models with GRPO-style RLVR on a curated 8-task rule-based reasoning dataset (RuleCollection-32K), using Domain-aware Dynamic Sampling (DADS) that reweights domains by historical rewards.

Problem

Rule-based reasoning with natural-language contextual rules varies widely in format, type, and complexity, and existing LRMs and RLVR recipes generalize poorly across these tasks while training multi-domain data imbalanced across domains is inefficient.

Approach

An online, batch-level domain sampler that uses exponentially smoothed domain rewards (no proxy models or extra rollouts) to softmax-reweight domains so under-optimized domains get more training samples, combined with KL-free, entropy-free GRPO and rule-shuffling regularization.

Key results

• RuleReasoner-8B reaches 84.0% average pass@1 on eight ID tasks, versus 79.9% for OpenAI o1 and 75.0% for GRPO.
• RuleReasoner-8B averages about 81.7% on three OOD benchmarks (BBH, ProverQA, BBEH), a 10.4-point gain over OpenAI o1 per the paper.
• RuleReasoner-4B achieves 78.3% average OOD pass@1 (7.3 points above o1), and DADS reaches OOD performance comparable to DAPO with roughly 1.4x fewer steps.

Surprising findings

• SFT matches RuleReasoner on ID tasks (81.9% vs 84.0%) but lags strongly on OOD (34.4 vs 54.5 average gain), indicating SFT generalizes poorly to unseen rules.
• Training on rule-only data (RuleCollection-32K) gives only 23.3% on AIME 2025, while AIME 1983-2024 alone gives 63.3%, which the authors note as a limitation of the rule data for math.

Limitations

• Scalability to larger models is unverified due to computational limits.
• Performance depends on rule-filtering quality; noisy or redundant rules hurt reasoning.
• Rule formats and complexity coverage remain limited relative to real-world rules; possible Qwen3 test-set leakage on BBH is noted.

Experimental setup

Datasets: ProofWriter ProntoQA Clutrr FOLIO LogicNLI AR-LSAT LogicalDeduction LogiQA BigBench-Hard BigBench-Extra-Hard ProverQA AIME 1983-2024

Benchmarks: Clutrr ProntoQA ProofWriter FOLIO LogicNLI AR-LSAT LogicalDeduction LogiQA BBH ProverQA BBEH AIME 2025

Metrics: pass@1 average accuracy across domains

Baselines: Hypotheses-to-Theories Chain-of-Logic Rule-Guided Feedback OpenAI o1 OpenAI o3-mini Claude-3.7-Sonnet DeepSeek-R1 SFT w/o CoT SFT w/ Short CoT SFT w/ Long CoT GRPO Dr. GRPO

Builds on: GRPO DAPO Qwen3 veRL AdaRFT

Closest papers by meaning

• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.90)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.89)
• REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient Reasoning (ICLR 2026 · similarity 0.89)
• Learn to Reason Efficiently with Adaptive Length-based Reward Shaping (ICLR 2026 · similarity 0.89)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.89)
• InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning (ICML 2026 · similarity 0.89)