AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Enhancing Language Model Reasoning with Structured Multi-Level Modeling

Siheng Xiong, Ali Payani, Faramarz Fekri

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a structured multi-level modeling approach to address long-horizon plan failures in long chain-of-thought reasoning of small language models, instead of a single outcome-reward RL policy.

Problem

Long-CoT inference-time scaling with a single outcome-reward RL policy suffers long-horizon plan failures where the implicit plan drifts from any valid strategy, especially severe for smaller LMs.

Approach

Separates reasoning into multiple levels of modeling rather than a single policy trained with outcome-reward RL, so that an explicit plan-level structure guides the long CoT (details not available in the provided abstract).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Step-DPO with TSMC process preferences instead of DPO in the multi-level model (ablation)MLR (Llama-3.1-8B) · average of MATH500/AIME24/GPQA/BoardGameQAAvg. Pass@1: 55.3 -> 66.1Improved
add high-level planner (multi-level modeling) on top of Step-DPO (ablation)Llama-3.1-8B · average of 4 benchmarksAvg. Pass@1: 59.1 -> 66.1Improved
add high-level planner on top of DPO (ablation)Llama-3.1-8B · average of 4 benchmarksAvg. Pass@1: 51.6 -> 55.3Improved
add iterative Step-DPO after SFT (ablation)MLR (Llama-3.1-8B) · average of 4 benchmarksAvg. Pass@1: 42.2 -> 66.1Improved
add iterative Step-DPO after SFT (ablation)MLR (Qwen-2.5-1.5B) · average of 4 benchmarksAvg. Pass@1: 35.8 -> 54.2Improved
use MLR instead of SFT + GRPO under same data budgetQwen-2.5-1.5B · average of 4 benchmarksAvg. Pass@1: 48.4 -> 54.2Improved
use MLR instead of SFT + GRPO under same data budgetLlama-3.1-8B · average of 4 benchmarksAvg. Pass@1: 60.0 -> 66.1Improved
use MLR instead of DeepSeek-R1-Distill with 10% SFT and 5% preference dataQwen-2.5-MATH-7B · average of 4 benchmarksAvg. Pass@1: 60.0 -> 66.2Improved
use Step-DPO instead of DPO on the SFT initialization (ablation)Llama-3.1-8B (low-level only) · average of 4 benchmarksAvg. Pass@1: 51.6 -> 59.1Improved
use Plan-and-Solve promptingQwen-2.5-1.5B · AIME24 Pass@1Pass@1: 3.3 (Instruct) -> 2.0Hurt
increase rollout count K in TSMC (ablation)iterative Step-DPO · preference estimationestimator variance and computeMixed
use small 1.5B SFT model as rollout policy instead of larger model (ablation)TSMC preference construction · preference qualitypreference accuracy and runtimeNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy