Research map / Large language models / Agents, code and math
Bridging the Gap between Source Code and Requirements Using GPT (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes GPT-Neo 2.7B to translate source code into natural-language requirements (SC2Req) and compares it against Basic GPT-3 and human rewriting on six software projects.
Problem
Legacy software systems often lack documentation, making reverse engineering and maintenance hard; automated code-to-requirements translation is needed to bridge the gap between source code and requirements.
Approach
Fine-tune a GPT-Neo 2.7B model on paired source code and requirement data (with multi-language data synthesized via GPT-3 translation) to generate requirements directly from code, evaluated with BLEU and semantic similarity.
Key results
- On Albergate (Java), SC2Req reaches BLEU 38.22 and semantic similarity 0.74 versus 18.74 and 0.58 for Basic GPT.
- On eTour, SC2Req (44.63 BLEU, 0.83 similarity) is close to Human Rewritten (45.37, 0.85).
- In the hold-out setting for Java, BLEU ranges from 27.07 (eTour) to 42.98 (EBT).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · Albergate (Java; training includes dataset) | BLEU: 18.74 -> 38.22 | Helped |
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · SMOS (Java; training includes dataset) | BLEU: 22.35 -> 45.73 | Helped |
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · EBT (Java; training includes dataset) | BLEU: 20.36 -> 48.85 | Helped |
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · eTour (Java; training includes dataset) | Semantic Similarity: 0.60 -> 0.83 | Helped |
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · iTrust (Java; training includes dataset) | Semantic Similarity: 0.57 -> 0.71 | Helped |
| Use SC2Req instead of Basic GPT | GPT-based code-to-requirements generation · eAnci (Java; training includes dataset) | Semantic Similarity: 0.62 -> 0.81 | Helped |
| Use SC2Req instead of Human Rewritten | Code-to-requirements generation · Albergate (Java; training includes dataset) | BLEU: 36.33 -> 38.22 | Helped |
| Use SC2Req instead of Human Rewritten | Code-to-requirements generation · SMOS (Java; training includes dataset) | BLEU: 47.38 -> 45.73 | Hurt |
| Use SC2Req instead of Human Rewritten | Code-to-requirements generation · EBT (Java; training includes dataset) | Semantic Similarity: 0.92 -> 0.90 | Hurt |
| Use SC2Req instead of Human Rewritten | Code-to-requirements generation · iTrust (Java; training includes dataset) | BLEU: 36.05 -> 37.21 | Helped |
| Train SC2Req without the held-out dataset | SC2Req · Albergate (Java; held out) | BLEU: 38.22 -> 33.05 | Mixed |
Surprising findings
- Human Rewritten outperforms SC2Req on several datasets such as SMOS, EBT and eTour in BLEU and similarity.
Limitations
- Multi-language datasets were synthetically generated by GPT-3 from Java code rather than collected from real projects.
- Performance is comparable to or below human rewriting on several datasets.
- Evaluation relies on BLEU and semantic similarity, with qualitative claims not systematically quantified.
Experimental setup
Datasets: Albergate SMOS EBT eTour iTrust eAnci
Benchmarks: Albergate SMOS EBT eTour iTrust eAnci
Metrics: BLEU semantic similarity
Baselines: Basic GPT (GPT-3 text-davinci-003) Human Rewritten
Builds on: GPT-Neo GPT-3
Closest papers by meaning
- MaintainCoder: Maintainable Code Generation Under Dynamic Requirements (NeurIPS 2025 · similarity 0.82)
- RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph (ICLR 2025 · similarity 0.82)
- NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents (ICML 2026 · similarity 0.82)
- Code Repair with LLMs gives an Exploration-Exploitation Tradeoff (NeurIPS 2024 · similarity 0.81)
- Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases (NeurIPS 2024 · similarity 0.81)
- RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms (NeurIPS 2026 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.