Research map / Multi-agent systems and games / LLM agents

DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DRAFT-RL, which combines Chain-of-Draft concise reasoning with multi-agent peer evaluation and a learned reward model to select drafts for PPO-plus-imitation training, reporting gains on code, math, and QA benchmarks.

Problem

LLM reasoning agents trained with RL often rely on single-shot responses and lack structured exploration and collaborative evaluation of diverse reasoning paths.

Approach

Each agent generates multiple Chain-of-Draft reasoning drafts per query, peer agents score them, a reward model selects the top draft, and the selected drafts drive PPO with an imitation loss.

Key results

• DRAFT-RL reports 82.6 MBPP and 87.6 HumanEval Pass@1, versus 78.1 and 84.5 for RLAIF (the strongest baseline in the table).
• On MATH, DRAFT-RL reports 55.8 versus 52.1 for RLAIF, while GSM8K is 94.2 versus 91.8.
• The paper claims 33-42% fewer training steps to convergence than RLHF/RLAIF.

Surprising findings

• Reported gains are large while the method adds per-query compute (about 19.4 s vs 8.9 s for CoD), and the paper also reports Claude-3.5-Sonnet as the only base, with agents differing only by 130M-parameter adapters on a closed model.

Limitations

• Relies on a closed API model (Claude-3.5-Sonnet) with adapters whose training details are sparse.
• Reward-model and peer-evaluation design, criteria, and statistical claims are described only at high level; the paper has inconsistent numbers (e.g., improvements cited as 2.4-4.5% and 3.5-3.7%) and reference placeholders throughout.
• Compute cost is higher than single-draft baselines due to multi-draft generation and peer evaluation.

Experimental setup

Datasets: MBPP HumanEval GSM8K MATH HotpotQA MMLU

Benchmarks: MBPP HumanEval GSM8K MATH HotpotQA MMLU

Metrics: Pass@1 accuracy EM F1 Likert human ratings convergence steps

Baselines: CoT CoD Self-Consistency ReAct Reflexion ChatEval RLHF RLAIF

Builds on: Chain-of-Draft PPO RLHF RLAIF Reflexion

Closest papers by meaning

• Reinforce LLM Reasoning through Multi-Agent Reflection (ICML 2025 · similarity 0.91)
• Tricks or Traps? A Deep Dive into RL for LLM Reasoning (ICLR 2026 · similarity 0.90)
• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.89)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• ReMA: Learning to Meta-Think for LLMs with Multi-agent Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.89)