Research map / Large language models / Agents, code and math

Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks

NeurIPS 2025 · Large language models › Agents, code and math · approach: Software engineering agents · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Code Graph Models (CGMs), which feed a repository code graph into a decoder LLM via a text encoder, adapter, and graph-aware attention mask, combined with an agentless Graph RAG pipeline, reaching 43.00% on SWE-bench Lite with open Qwen2.5-72B.

Problem

Repository-level software engineering tasks (issue fixing, code completion) remain hard for open-source LLMs, which currently rely on proprietary agents; existing graph-enhanced code models flatten graph context into text and handle function/file-level tasks only.

Approach

Represent repositories as text-rich code graphs; encode node text with CodeT5+ and project it via an MLP adapter into LLM node tokens, and replace the causal mask between node tokens with the graph adjacency mask so attention follows graph edges (GNN-like message passing), then fine-tune with LoRA and pretrain via subgraph-to-code reconstruction; pair with a four-module agentless Graph RAG (Rewrit

Key results

• CGM-SWE-PY with Qwen2.5-72B and agentless Graph RAG reaches 43.00% resolution on SWE-bench Lite, first among open-weight models and eighth overall, 12.33 points above Moatless+DeepSeek-V3.
• CGM-Multi reaches 14.29% on SWE-bench-java Verified, above SWE-agent variants with DeepSeek-V2 (9.89%) and GPT-4o (6.59%).
• On CrossCodeEval Python, CGM-Multi with Qwen2.5-Coder-7B reaches 46.23 EM vs 43.15 for BM25 baseline with the same model.

Measured effects reported in the paper

Surprising findings

• Removing the Reranker drops SWE-bench Lite resolution by 24.67 points, the largest drop among RAG components.
• Feeding the unfiltered full repository graph (w/o R3) drops performance by 33.33 points, showing that retrieval filtering matters more than raw graph context.

Limitations

• Only Python and Java are supported by the code graph.
• Gap of roughly 10 points to top closed-source agent systems on SWE-bench Lite remains.
• Multi-task CGM-Multi trails the Python-specialized model on SWE-bench Lite.

Experimental setup

Datasets: SWE-bench training set GitHub issue-patch pairs GitHub code graphs from 20k repositories CrossCodeEval ComplexCodeEval

Benchmarks: SWE-bench Lite SWE-bench Verified SWE-bench-java Verified CrossCodeEval ComplexCodeEval

Metrics: resolve rate exact match edit similarity recall

Baselines: Agentless Moatless Lingma SWE-GPT SWE-Fixer SWE-agent RLCoder R2C2 RepoFuse BM25 Mistral-Large-2

Builds on: GraphCodeBERT Agentless LoRA CodeT5+ DeepSeek-Coder

Closest papers by meaning

• RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph (ICLR 2025 · similarity 0.89)
• Nemotron-CORTEXA: Enhancing LLM Agents for Software Engineering Tasks via Improved Localization and Solution Diversity (ICML 2025 · similarity 0.87)
• CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks (NeurIPS 2025 · similarity 0.86)
• SWE-bench: Can Language Models Resolve Real-world Github Issues? (ICLR 2024 · similarity 0.86)
• MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution (NeurIPS 2024 · similarity 0.86)
• CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules (ICLR 2024 · similarity 0.85)