Research map / Large language models / Reasoning and chain of thought

CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules

ICLR 2024 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CodeChain, an inference-time framework that prompts LLMs to write modular code via chain-of-thought and iteratively self-revises using clustered representative sub-modules, improving APPS and CodeContests pass@1.

Problem

LLMs struggle with complex competitive programming tasks, largely because they generate monolithic code rather than decomposing it into reusable sub-modules; prior self-revision methods use only per-sample feedback.

Approach

Chain-of-thought prompting elicits modular code with explicit sub-module signatures; sub-modules from all samples are embedded, clustered, and the centroid implementations are fed back into the prompt for iterative self-revision rounds.

Key results

• CodeChain+GPT3.5 reaches 30.24% pass@1 on APPS test (vs 22.33% for GPT3.5 direct generation), a new SoTA reported in the paper.
• CodeChain+WizardCoder-15B reaches 10.50% pass@1 on APPS test vs 7.90% for WizardCoder.
• With synthetic-test CodeT filtering, CodeChain+GPT3.5 reaches 35.34% pass@1 on APPS test; CodeChain+GPT4 reaches 61.50% on a 20-sample subset, above Self-repair+GPT4 with GPT4 feedback (33.30%).

Measured effects reported in the paper

Surprising findings

• Chain-of-thought modular prompting hurts direct-generation correctness (e.g., 22.43 vs 26.00 All on APPS validation) but gains outweigh this after self-revision.
• Clustering alone is insufficient; filtering sub-modules by public/private tests before clustering gives further gains, and synthetic-test filtering can hurt performance versus no filtering.

Limitations

• Gains depend on high-quality test cases for filtering; synthetic tests are noisy and can degrade results.
• Performance drops after the optimal revision round may come from overfitting to the small public test set.
• Evaluation on a small 20-sample subset for the Self-repair comparison and on a 39-problem LeetCodeHardGym setup that differs from the original.

Experimental setup

Datasets: APPS CodeContests LeetCodeHardGym

Benchmarks: APPS CodeContests LeetCodeHardGym

Metrics: pass@1 pass@5 relative pass@1 improvement Likert modularity/reusability score

Baselines: Codex CodeT5 CodeRL+CodeT5 text-davinci-002 Self-edit+text-davinci-002 code-davinci-002 WizardCoder GPT3.5 Self-repair+GPT4 Self-debug Reflexion CodeT

Builds on: chain-of-thought prompting CodeT Self-debug Self-repair Reflexion

Closest papers by meaning

• Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain (ICLR 2024 · similarity 0.89)
• CodeSync: Synchronizing Large Language Models with Dynamic Code Evolution at Scale (ICML 2025 · similarity 0.89)
• Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding? (ICML 2025 · similarity 0.89)
• BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.88)
• Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation (NeurIPS 2024 · similarity 0.88)
• CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance (ICML 2025 · similarity 0.88)