Research map / Large language models / Agents, code and math

LeAct: Learning to Reason from Expert Actions

NeurIPS 2026 · Large language models › Agents, code and math · approach: Reasoning benchmarks and platforms · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LeAct, which recovers latent natural-language chain-of-thought from silent expert actions (CFR solvers, DeepCFR, frontier trajectories) by sampling backward CoT candidates and keeping those that raise the student's probability of the expert action.

Problem

Expert systems (game solvers, planners) produce near-optimal actions without reasoning traces, so student LLMs lack CoT supervision from them; naive CoT generation suffers from rationalization with no causal link to the action.

Approach

Treat the CoT as a latent variable; sample backward CoT candidates conditioned on the expert action, score each by the forward delta (increase in student's log-probability of the expert action vs. a self-baseline), keep top-K positive ones, and train forward (CoT then expert policy) and backward (CoT given redacted policy) jointly via SFT, derived as an approximate IWAE/variational-EM M-step.

Key results

• On Flop Hold'em, LeAct wins head-to-head against NoCoT BC by +60 mbb/g (±13) over 50K hands.
• On Leduc 10r4s rank-split held-out ranks, LeAct single-sample KL 0.716 vs 2.071 for CoT ExIt.
• On BuilderBench OOD partition, LeAct reaches 0.5777 progress vs 0.4361 for CoT ExIt and 0.4815 coldstart.

Surprising findings

• Random ranking of CoT candidates erases the advantage, showing the selection filter is load-bearing.
• Off-diagonal pairings of method and supervision target (e.g., LeAct with model's own action) collapse to no gain in poker.

Limitations

• Single fixed 8B student backbone; behavior at frontier-scale students untested.
• Scoring relies on parsed textual policy line rather than token-level likelihood, a proxy with 89-98% top-1 agreement.
• Absolute values are not comparable across games; BuilderBench uses a heuristic action-match reward rather than KL.

Experimental setup

Datasets: Leduc Hold'em CFR solutions Liar's Dice CFR solutions 3-Player Leduc CFR solutions Flop Hold'em DeepCFR teacher labels BuilderBench frontier-LLM trajectories

Benchmarks: Leduc Hold'em exploitability Liar's Dice exploitability 3-Player Leduc NashConv Flop Hold'em KL and mbb/g BuilderBench cube-placement progress

Metrics: exploitability NashConv KL divergence mbb/g win rate cube-placement progress

Baselines: NoCoT BC CoT ExIt coldstart gpt-5.5

Builds on: expert iteration importance-weighted autoencoder reweighted wake-sleep variational EM DeepCFR

Closest papers by meaning

• Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.87)
• LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning (NeurIPS 2026 · similarity 0.87)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.87)
• SuCo: Sufficiency-guided Continuous Adaptive Reasoning (ICML 2026 · similarity 0.87)
• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.87)
• Thinkless: LLM Learns When to Think (NeurIPS 2025 · similarity 0.87)