Research map / Large language models / Language, safety and interpretability

Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates

ICML 2024 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CACP, a black-box counterfactual testing framework that applies predicate-specific code mutations to evaluate whether ten large code models understand programming concepts (control flow, data flow, data types, identifier naming) on code completion.

Problem

It is unclear whether large code models understand the logical programming concepts (program predicates) underlying the programs they complete, and end-to-end benchmarks do not attribute failures to specific concepts.

Approach

Generate minimal counterfactual code pairs via semantics-preserving mutations (if-else flip, independent statement swap, def-use break, variable renaming/shuffling) and measure the Average Mutation Effect on unit-test pass/fail, requiring only hard-label black-box access.

Key results

• Variable-name and if-else-flip mutations yield AMEs as high as about 33-34% across models, indicating gaps in understanding of naming and control flow.
• Independent-swap is the best-understood predicate, with AMEs typically in the 2-7% range for larger models such as GPT-4 and PaLM 2 (340B).
• Larger models and code-specific training/fine-tuning generally reduce variable-name AME; StarCoder's if-else flip AME correlates with operator frequency in its training data.

Measured effects reported in the paper

Surprising findings

• AME correlates with how frequently relational operators appear in StarCoder's training data, suggesting memorization contributes to mutation effects.
• Code fine-tuning does not always reduce mutation effects for control-flow and data-flow predicates.

Limitations

• Crafting semantics-preserving mutations requires substantial manual effort and domain knowledge.
• Def-Use Break yields very few valid counterfactual pairs (22 for HumanEval+MBPP), limiting statistical power.
• Evaluation is restricted to Python and test-based attribution functions.

Experimental setup

Datasets: HumanEval MBPP CodeContests HumanEvalPack

Benchmarks: HumanEval MBPP CodeContests HumanEvalPack

Metrics: Average Mutation Effect (AME) unit-test correctness original accuracy

Baselines: Llama 2 Code Llama PaLM 2 PaLM 2-S* GPT-4

Builds on: HumanEval MBPP CodeContests HumanEvalPack

Closest papers by meaning

• CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs (ICLR 2025 · similarity 0.88)
• Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding? (ICML 2025 · similarity 0.86)
• CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks (NeurIPS 2025 · similarity 0.86)
• What Makes Large Language Models Reason in (Multi-Turn) Code Generation? (ICLR 2025 · similarity 0.86)
• CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning (NeurIPS 2025 · similarity 0.86)
• NExT: Teaching Large Language Models to Reason about Code Execution (ICML 2024 · similarity 0.86)