Research map / Label-efficient and robust learning / Recognition and long-tail
Break-It-Fix-It: Unsupervised Learning for Program Repair
Michihiro Yasunaga, Percy Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BIFI: add critic-verified fixer outputs on real bad code and breaker-generated bad examples | Initial fixer on GitHub-Python · GitHub-Python test set | repair accuracy: 62.0% -> 90.5% | Improved |
| Use BIFI on the DrRepair initialization | DrRepair · DeepFix test set | repair accuracy: 66.1% -> 71.7% | Improved |
| Add real bad examples to fixer training without a learned breaker | Initial fixer · GitHub-Python test set | repair accuracy: 62.0% -> 88.6% | Improved |
| Add breaker-generated bad examples alongside real bad examples | FixerOnly on GitHub-Python · GitHub-Python test set | repair accuracy: 88.6% -> 90.5% | Improved |
| Add breaker-generated bad examples when real bad data is limited | FixerOnly on GitHub-Python · GitHub-Python, Bad 10% | repair accuracy: 78.5% -> 86.7% | Improved |
| Train on a small amount of real bad examples instead of relying on synthetic bad examples only | FixerOnly · GitHub-Python, Bad 10% | repair accuracy: 62.7% -> 78.5% | Improved |
| Remove real bad examples from fixer training (ablation) | BIFI · GitHub-Python test set | repair accuracy: 90.5% -> 84.6% | Hurt |
| Remove critic verification of fix and break attempts (ablation) | BIFI · GitHub-Python test set | repair accuracy: 90.5% -> 84.0% | Hurt |
| Remove both critic verification and real bad examples, as in backtranslation (ablation) | BIFI · GitHub-Python test set | repair accuracy: 90.5% -> 80.1% | Hurt |
| Continue training the fixer using synthetic perturbations only (ablation) | Initial fixer · GitHub-Python test set | repair accuracy | No clear effect |
| Reduce available real bad examples from Bad 100% to Bad 50% (ablation) | BIFI · GitHub-Python test set | repair accuracy: 90.5% -> 89.0% | Hurt |
| Reduce available real bad examples from Bad 100% to Bad 10% (ablation) | BIFI · GitHub-Python test set | repair accuracy: 90.5% -> 86.7% | Hurt |
Closest papers by meaning
- Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization (AAAI 2025 · similarity 0.86)
- AuPair: Golden Example Pairs for Code Repair (ICML 2025 · similarity 0.85)
- Less Is More: Adaptive Program Repair with Bug Localization and Preference Learning (AAAI 2025 · similarity 0.84)
- Code Repair with LLMs gives an Exploration-Exploitation Tradeoff (NeurIPS 2024 · similarity 0.83)
- Is Self-Repair a Silver Bullet for Code Generation? (ICLR 2024 · similarity 0.82)
- LeDex: Training LLMs to Better Self-Debug and Explain Code (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
