Research map / Large language models / Instruction and fine-tuning
CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a Llama-3.1-8B-based guide (CodeSteerLLM) with multi-turn SFT and DPO on synthesized SymBench trajectories to steer GPT-4o's code/text generation across 37 symbolic tasks.
Problem
LLMs struggle to decide when to reason in text versus generate code for symbolic tasks, and existing steering methods (Code Interpreter, AutoGen, multi-agent summarization) underuse symbolic computation.
Approach
A small fine-tuned guide model reviews the TaskLLM's answers over multiple turns and provides code/text guidance, trained with multi-turn SFT (double weight on final two turns) and tree-sampled multi-turn DPO scored by a turn-penalized performance score, plus symbolic and self-answer checkers.
Key results
- GPT-4o + CodeSteer raises average normalized score on SymBench from 53.3 to 86.4, versus o1 at 82.7, DeepSeek R1 at 76.8 and o1-preview at 74.8.
- CodeSteer guidance transferred to Claude-3-5-Sonnet, Mistral-Large and GPT-3.5 on five tasks, with Claude average rising from 29.1 to 92.0.
- o1 + CodeSteer improves o1 on five unseen tasks, e.g., Synthesis Decomposition from 57 to 94 and Combinatorial Calculation from 57 to 95.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CodeSteer guidance to GPT-4o | GPT-4o · SymBench total | Average normalized score: 53.3 -> 86.4 | Helped |
| Add CodeSteer guidance to GPT-4o | GPT-4o · SymBench unseen | Average normalized score: 72.1 -> 81.3 | Helped |
| Add CodeSteer guidance | Claude-3-5-Sonnet · Five representative tasks | Average normalized score: 29.1 -> 92.0 | Helped |
| Add CodeSteer guidance | Mistral-Large · Five representative tasks | Average normalized score: 31.0 -> 59.8 | Helped |
| Add CodeSteer guidance | GPT-3.5 · Five representative tasks | Average normalized score: 8.6 -> 42.3 | Helped |
| Add CodeSteer guidance | o1 · Cryptanalysis | Task success rate (%): 60 -> 73 | Helped |
| Add DPO fine-tuning (ablation) | CodeSteer SFT model · SymBench total | Average normalized score: 79.1 -> 86.4 | Helped |
| Add last-two-turn data augmentation (ablation) | CodeSteer SFT model without DPO · SymBench total | Average normalized score: 77.6 -> 79.1 | Helped |
| Add Symbolic Checker (ablation) | CodeSteer · SymBench total | Average normalized score: 77.3 -> 86.4 | Helped |
| Add Self-answer Checker (ablation) | CodeSteer · SymBench total | Average normalized score: 75.0 -> 86.4 | Helped |
| Add Symbolic Checker (ablation) | Symbolic Agent · SymBench total | Average normalized score: 69.5 -> 74.8 | Helped |
| Add Self-answer Checker (ablation) | Symbolic Agent · SymBench total | Average normalized score: 67.0 -> 74.8 | Helped |
Surprising findings
- GPT-4o Code Interpreter fails on medium-difficulty questions because it is overconfident and answers with text.
- Prompt-based heuristics (Few-Shot, Code-First-Rule, Code-First-Agent) perform significantly worse than the trained CodeSteer guide.
Limitations
- TaskLLM capability limits can cause failures regardless of code or text choice.
- Generated code is sometimes inefficient and times out on larger instances.
- Performance drops on medium-complexity samples, where CodeSteer sometimes picks text reasoning and errs.
Experimental setup
Datasets: SymBench (synthesized) GSM-Hard MATH Big-Bench Hard tasks LogicGame tasks
Benchmarks: SymBench
Metrics: task success rate average normalized score
Baselines: o1 o1-preview DeepSeek R1 Code Interpreter AutoGen Conca. Code/Text Choice Code + Text + Sum.2 Symbolic Agent All Text + CoT All Code + CoT
Builds on: DPO AutoGen Code Interpreter LogicGame
Closest papers by meaning
- Steering Large Language Models between Code Execution and Textual Reasoning (ICLR 2025 · similarity 0.90)
- rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.90)
- CodePlan: Unlocking Reasoning Potential in Large Language Models by Scaling Code-form Planning (ICLR 2025 · similarity 0.88)
- AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data (NeurIPS 2024 · similarity 0.88)
- CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules (ICLR 2024 · similarity 0.88)
- CRANE: Reasoning with constrained LLM generation (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.