Research map / Large language models / Agents, code and math
CursorCore: Assist Programming through Aligning Anything
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the CursorCore framework (History/Current/User/Assistant conversation format), the Assist Programming Eval benchmark, and the AIPC data pipeline, and fine-tunes Deepseek-Coder, Yi-Coder and Qwen2.5-Coder base models on 219K synthesized samples to assist programming.
Problem
Existing code LLM assistants (completion, insertion, instruction-based editing) do not jointly use coding history, current code context, and user instructions, and lack automated training data for the full programming process.
Approach
A conversation-style framework that aligns models with programming history, current code, and user instructions to predict code edits, plus a data pipeline that synthesizes programming-process traces from AI programmer simulation, git commits and online judge submissions.
Key results
- CursorCore-QW2.5-7B achieves 55.5 average Pass@1 on Assist Programming Eval, the best among 6B+ models, exceeding the next best by 4.3%.
- CursorCore-QW2.5-1.5B reaches 50.0 average on Assist Programming Eval, the best among 1B+ models, surpassing other models by 10.4%.
- GPT-4o remains top overall at 65.2 average, with open models such as Qwen2.5-72B-Instruct close at 65.2 average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Qwen2.5-Coder-7B on CursorCore training data | Qwen2.5-Coder-7B · Assist Programming Eval Python | Pass@1 Avg.: 48.8 -> 55.5 | Helped |
| Fine-tune Qwen2.5-Coder-1.5B on CursorCore training data | Qwen2.5-Coder-1.5B · Assist Programming Eval Python | Pass@1 Avg.: 39.6 -> 50.0 | Helped |
| Use AI Programmer data rather than the other individual data sources (ablation) | Deepseek-Coder-1.3B · Assist Programming Eval | Pass@1 | Helped |
| Add AI Programmer data to Evol-Instruct data (ablation) | Deepseek-Coder-1.3B · Assist Programming Eval | Pass@1 | Helped |
| Train on Evol-Instruct alone instead of AI Programmer data (ablation) | Deepseek-Coder-1.3B · Assist Programming Eval | Pass@1 | Hurt |
| Train using multilingual data rather than only Python data (ablation) | CursorCore · Assist Programming Eval | Pass@1 | Hurt |
| Represent code modifications using whole-file format instead of unified diff (ablation) | Deepseek-Coder-1.3B · Assist Programming Eval | performance | Helped |
| Represent code modifications using unified diff rather than whole-file format (ablation) | Deepseek-Coder-1.3B · Assist Programming Eval | performance | Hurt |
| Add location information to the input (ablation) | CursorCore-DS-1.3B · Assist Programming Eval | performance | No clear effect |
| Add a reasoning process to training outputs (ablation) | CursorCore · Assist Programming Eval | performance | Helped |
| Add a sliding-window limit to historical editing records (ablation) | CursorCore · Assist Programming Eval | performance | No clear effect |
| Use the Tab format instead of Chat or Inline format (ablation) | CursorCore · EvalPlus, HumanEvalFix | performance | Mixed |
Surprising findings
- Qwen2.5-Coder base model outperforms its instruction-tuned variant on Assist Programming Eval.
- The whole-file edit format performs best, while the unified diff format performs worst despite being common.
Limitations
- 6B+ model underperforms its comparable models on several benchmarks, attributed to insufficient high-quality data.
- Reasoning-augmented outputs add much output length with only slight gains, so not used in final models.
- Benchmark is ~1K samples and mostly short, single-file tasks; location information has minimal effect on it.
Experimental setup
Datasets: The Stack OSS-Instruct EditPackFT CodeNet Evol-Instruct HumanEval
Benchmarks: Assist Programming Eval EvalPlus (HumanEval+, MBPP+) CanItEdit OctoPack HumanEvalFix
Metrics: Pass@1
Baselines: GPT-4o Qwen2.5-Coder-7B-Instruct DeepSeek-Coder-6.7B-Instruct Yi-Coder-9B-Chat Qwen2.5-72B-Instruct Mistral-Large-Instruct DeepSeek-Coder-V2-Instruct
Builds on: HumanEval Evol-Instruct OSS-Instruct EvalPlus CanItEdit
Closest papers by meaning
- CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance (NeurIPS 2025 · similarity 0.85)
- SelfCodeAlign: Self-Alignment for Code Generation (NeurIPS 2024 · similarity 0.85)
- Copilot Arena: A Platform for Code LLM Evaluation in the Wild (ICML 2025 · similarity 0.85)
- StackEval: Benchmarking LLMs in Coding Assistance (NeurIPS 2024 · similarity 0.85)
- BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.84)
- JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.