AI Paper Atlas

Research map / Large language models / Agents, code and math

CodeClash: Benchmarking Goal-Oriented Software Engineering

John Yang, Kilian Lieret, Joyce Yang, Carlos E. Jimenez, Muhtasham Oblokulov, Aryan Siddiqui, Ofir Press, Ludwig Schmidt, Diyi Yang

ICML 2026 · Large language models › Agents, code and math · approach: Software engineering agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
show players a read-only copy of opponent's previous code (ablation)GPT-5 · Core Warround win rate: +7.8% (74.6%)Improved
show players a read-only copy of opponent's previous code (ablation)Claude Sonnet 4.5 · Core Warround win rate: -1.8% (53.2%)No clear effect
show players a read-only copy of opponent's previous code (ablation)Gemini 2.5 Pro · Core Warround win rate: -5.5% (22.7%)Hurt
increase from 2 to 6 players per tournament (ablation)Core War tournaments · Core Warlead changes: 18.2% -> 48.4%Improved
replace mini-SWE-agent with SWE-agent scaffold (ablation)three models · CC:Ladderladder rank: at most 2 ranks changeNo clear effect
randomly shuffle player order (ablation)MIT Battlecode 2025 arena · dummy-player tournamentswin rate: positional advantage -> 50% win rateImproved
lose several consecutive roundsClaude Sonnet 4.5 · CodeClashprobability of winning next round: 71% overall -> below one third after one lossHurt
lose a single roundo3 · CodeClashprobability of winning next round: 65% -> 26%Hurt
more edit-behavior (files edited, edit size, turns)8 LMs · CodeClashwin rateNo clear effect
more frequent inspection of opponent codeGPT-5 vs Claude Sonnet 4.5 · Core Warwin rate: 12.8% vs 99.3% accessNo clear effect
validate edits with arena simulations or testsClaude Sonnet 4.5 · CodeClashfraction of rounds validated: 56%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy