AI Paper Atlas

Research map / Large language models / Agents, code and math

Is Code Better Than Language for Algorithmic Reasoning?

Terry Tong, Yu Feng, Surbhi Goel, Dan Roth

ICML 2026 · Large language models › Agents, code and math · approach: Reasoning benchmarks and platforms · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
execute generated code with external Python runtime instead of LLM simulation (Route 2 -> Route 3)code-trace reasoning · 40-task algorithmic setaccuracy: 17.37 -> 48.84 (+31.47pp)Improved
use code traces instead of natural-language traces with LLM execution (Route 1 -> Route 2)LLM reasoning · 40-task algorithmic setaccuracy: 17.21 -> 17.37 (+0.15pp)No clear effect
add external execution to code traces for GPT-4o-miniGPT-4o-mini · 40-task set, per modelaccuracy: 12.9 -> 49.1Improved
use code traces instead of NL for GPT-4o-miniGPT-4o-mini · 40-task set, per modelaccuracy: 14.1 -> 12.9Hurt
use code traces instead of NL for Gemini 2.0 FlashGemini 2.0 Flash · 40-task set, per modelaccuracy: 17.5 -> 18.1No clear effect
add external execution on O(n^3) and O(n log n) tasksRoute 3 · complexity strataaccuracy: 0.0 -> 0.0No clear effect
translate code traces to NL with more in-context shots (0 -> 5) (ablation)translated NL trace, Claude Haiku 4.5 · Translation shot ablation (25% subset)accuracy gain over question-only baseline: +9.70pp -> +17.30ppImproved
use translated NL instead of native NL trace (5 shots) (ablation)Claude Haiku 4.5 · Translation shot ablation (25% subset)accuracy: 55.70% -> 50.00%Hurt
add reasoning trace (native NL) to question-only prompt (ablation)Claude Haiku 4.5 · Translation shot ablation (25% subset)accuracy: 32.70% -> 55.70%Improved
RLM execution: code arm instead of NL arm for Claude Haiku 4.5Claude Haiku 4.5 · RLM 25% subset, seed 0accuracy: 30.9% -> 29.4%Hurt
RLM execution: code arm instead of NL arm for Gemini 2.5 ProGemini 2.5 Pro · RLM 25% subset, seed 0accuracy: 45.2% -> 47.5%Improved
code execution instead of simulation for Claude Opus 4.6Claude Opus 4.6 · Frontier controlled subsetaccuracy: 73.42% -> 77.54%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy