Research map / Large language models / Reasoning and chain of thought

Reverse-Engineered Reasoning for Open-Ended Generation

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes REverse-Engineered Reasoning (REER), which synthesizes deep thinking trajectories for open-ended writing by perplexity-guided gradient-free local search from known-good outputs, releases the 20K DeepWriting-20K dataset, and fine-tunes Qwen3-8B-Base into DeepWriter-8B.

Problem

Instilling deep reasoning for open-ended, non-verifiable creative generation, where RL lacks clear reward signals and distillation is costly and capped by the teacher.

Approach

Work backwards from a high-quality reference output: iteratively refine an initial thinking trajectory segment by segment, selecting candidates that minimize the perplexity of the known solution given query and trajectory, then fine-tune a base model on the resulting (query, trajectory, answer) triples.

Key results

• DeepWriter-8B scores 91.28 on LongBench-Write, above GPT-4o (83.1) and Claude 3.5 (89.3), and below Claude 3.7 (97.8).
• DeepWriter-8B averages about 18 points above LongWriter-8B on WritingBench domains (e.g., WB-A 72.20 vs 57.97).
• Removing the synthesized data drops HB-B from 87.48 to 73.73; removing iterative search lowers WB-A from 72.20 to 66.72.

Surprising findings

• DeepWriter-8B outperforms GPT-4o and Claude 3.5 on LongBench-Write despite being an 8B model.
• Removing Literature & Arts data degrades performance across all benchmarks, including technical WritingBench domains.

Limitations

• Relies on LLM judges (Claude-3.7, GPT-4o) with acknowledged bias; WritingBench protocol discrepancy with original paper.
• Still trails Claude 3.7 on several benchmarks, especially depth of analysis.
• Training data includes GPT-4o-generated queries and relies on a Qwen2.5-32B generator, so the approach still depends on strong models.

Experimental setup

Datasets: WildChat LongWriter6K OpenThoughts Project Gutenberg r/WritingPrompts

Benchmarks: LongBench-Write HelloBench WritingBench

Metrics: LLM-judge score ablation scores normalized rubric scores on five deep-thinking dimensions

Baselines: GPT-4o Claude 3.5 Claude 3.7 LongWriter-8B

Builds on: LongWriter OpenThoughts Monte Carlo Tree Search VeriFree

Closest papers by meaning

• Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcement (AAAI 2026 · similarity 0.86)
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (ICLR 2026 · similarity 0.85)
• RAST: Reasoning Activation in LLMs via Small-model Transfer (NeurIPS 2025 · similarity 0.85)
• SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond (NeurIPS 2025 · similarity 0.85)