AI Paper Atlas

Research map / Large language models / Agents, code and math

BEST: Benchmarking Efficiency in Space and Time for LLM-Generated Code

Aocheng Shen, Boyu Zhang, Jiaze Li, Ruixuan Ma, Qiankun Zhang, Jing Wang, Bin Yuan, Shenghao Liu, Xianjun Deng

ICML 2026 · Large language models › Agents, code and math · approach: Code generation benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
measure dual efficiency (time and space) instead of correctness only50 LLMs on BEST · BESTdual@10 vs pass@10: GPT-5.2: pass@10 0.879 -> dual@10 0.811Hurt
use reasoning (thinking) mode in the LLM (ablation)DeepSeek-V3.2 · BEST (tau=sigma=1.2)dual@10: 0.630 -> 0.678Improved
use reasoning (thinking) mode in the LLM (ablation)DeepSeek-V3.2 · BEST (tau=sigma=1.2)dual@1: 0.510 -> 0.508No clear effect
measure space-only efficiency instead of time-only efficiencyLLMs generally · BESTdual@10: Llama-3.1-8B time-only 0.244 vs space-only 0.248 (the only exception)Hurt
sample 10 codes instead of 1 (temperature 0.8) (ablation)all LLMs average · BESTpass@k: 0.351 -> 0.495Improved
sample 10 codes instead of 1 (temperature 0.8) (ablation)all LLMs average · BESTdual@k: 0.250 -> 0.349Improved
newer model release within DeepSeek familyDeepSeek models · BEST (tau=sigma=1.2)dual@10: DeepSeek-Coder-33B-Instruct 0.246 -> DeepSeek-V3.2-Thinking 0.678Improved
larger model size within the Llama family (ablation)Llama-3.1 · BEST (tau=sigma=1.2)dual@10: 0.194 -> 0.410Improved
use instruction-tuned instead of base variant (ablation)DeepSeek-Coder-33B · BEST (tau=sigma=1.2)dual@10: 0.216 -> 0.246Improved
better correctness of the modelCodeGemma-7B vs Phind-CodeLlama-34B-V1 · BESTdual@10Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy