Research map / Large language models / Language, safety and interpretability
Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach
Yuanheng Li, Zhuoyang Chen, Xiaoyun Liu, Yuhao Wang, Mingwei Liu, Yang Shi, Kaifeng Huang, Shengjie Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune syntax-determined tokens when scoring membership (SynPrune vs best baseline DC-PDD) | MIA on Pythia 2.8B · Python function MIA benchmark (1:1) | AUROC (%): 50.1 -> 61.3 | Improved |
| prune syntax-determined tokens (SynPrune vs LOSS) | MIA on GPT-J 6B · Python function MIA benchmark (1:1) | AUROC (%): 43.3 -> 59.7 | Improved |
| prune syntax-determined tokens at 5:1 member-heavy ratio | MIA on GPT-J 6B · Python function MIA benchmark (5:1) | AUROC (%): 43.6 -> 63.1 | Improved |
| add compound-statement syntax conventions to pruning (ablation) | SynPrune on StableLM-Alpha 3B · Python function MIA benchmark (1:1) | AUROC (points): -7.1 when removed | Improved |
| add data-model syntax conventions to pruning (ablation) | SynPrune on Pythia 2.8B · Python function MIA benchmark (1:1) | AUROC (points): -5.1 when removed | Improved |
| add expression syntax conventions to pruning (ablation) | SynPrune on Pythia 2.8B · Python function MIA benchmark (1:1) | AUROC (points): -4.5 when removed | Improved |
| add single-statement syntax conventions to pruning (ablation) | SynPrune on GPT-J 6B · Python function MIA benchmark (1:1) | AUROC (points): -1.6 when removed | Improved |
| add data-model conventions to pruning in GPT-Neo at 1:5 (ablation) | SynPrune on GPT-Neo 2.7B · Python function MIA benchmark (1:5) | AUROC (points): -0.1 when removed | No clear effect |
| evaluate on long instead of short member functions | SynPrune on GPT-J 6B · Python function MIA benchmark (member functions by length) | false negative rate (%): 0.00 -> 87.09 | Hurt |
| evaluate on long instead of short member functions | SynPrune on Pythia 2.8B · Python function MIA benchmark (member functions by length) | false negative rate (%): 2.10 -> 75.12 | Hurt |
Closest papers by meaning
- ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding (ICLR 2025 · similarity 0.88)
- Security Attacks on LLM-based Code Completion Tools (AAAI 2025 · similarity 0.87)
- ProSec: Fortifying Code LLMs with Proactive Security Alignment (ICML 2025 · similarity 0.86)
- Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting (AAAI 2025 · similarity 0.86)
- Combating Data Laundering in LLM Training (NeurIPS 2026 · similarity 0.86)
- Evaluation of Large Language Models on Code Obfuscation (Student Abstract) (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
