Research map / Large language models / Language, safety and interpretability

Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Data and evaluation methodology · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ProRec, a probe-and-recover framework where a CodeArt binary encoder aligned to a frozen Code Llama SCFM synthesizes source-code fragments that black-box LLMs use as context for binary summarization and function name recovery.

Problem

Human-oriented binary reverse engineering (lifting stripped binary/decompiled code to human-readable, source-like content such as summaries and function names) suffers from the binary-source semantic gap; existing supervised small models generalize poorly and direct LLM prompting struggles on stripp

Approach

Align a binary function encoder (CodeArt) to a frozen source code foundation model via a small MLP prefix so the SCFM, used as a parametric knowledge base, generates symbol-rich source fragments from binary input; a black-box LLM then analyzes these probed contexts plus the decompiled code to recover the target output.

Key results

• ProRec with GPT-3.5-turbo raises CHRF from 30.4 to 33.5 (+3.1 points, ~10.3% relative) and G4-F from 3.6 to 4.2 on summarization.
• On function name recovery with GPT-3.5-turbo, ProRec improves SymLM precision from 16.3 to 22.2 and recall from 20.9 to 28.3 (zero-shot baseline vs ProRec).
• Human study (50 summaries per method, GPT-3.5-turbo) gives context relevance 4.76 for ProRec vs 4.29 decompiled-only and 4.49 retrieval; GPT4Evaluator has highest Spearman correlation with humans (0.58).

Measured effects reported in the paper

Surprising findings

• Retrieval augmentation with a strong cross-modal dense retriever sometimes degrades results, e.g., Gemini-Pro CHRF drops from 27.1 to 26.4 with retrieval.
• Larger base SCFM lowers prober validation loss and raises n-gram overlap, yet CHRF on summarization does not significantly improve, suggesting modest-size SCFMs suffice.

Limitations

• Only intra-procedural analysis; does not handle full binaries with multiple functions or call graphs.
• Simple prober architecture and alignment objective may not be optimal.
• Failure cases where generated contexts are relevant but the recovered function name still differs from source (e.g., missing the 'temperature sensor' association).

Experimental setup

Datasets: GHCC-compiled GitHub repositories (270k binary-source function pairs) 1k test samples subset

Benchmarks: binary summarization (CHRF, GPT4-based functionality and context relevance metrics) binary function name recovery (SymLM token-level P/R/F1, character-level BLEU/ROUGE-L/METEOR)

Metrics: CHRF G4-F G4-C SymLM precision SymLM recall SymLM F1 cBLEU cRoL cMETEOR

Baselines: GPT-3.5-turbo decompiled-only prompting retrieval-augmented baseline with dense retriever CP-BCS self-probing

Builds on: CodeArt Code Llama CodeT5+ retrieval-augmented generation GenRead

Closest papers by meaning

• Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation (NeurIPS 2025 · similarity 0.84)
• Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting (AAAI 2025 · similarity 0.84)
• Towards A Generalist Code Embedding Model Based On Massive Data Synthesis (NeurIPS 2025 · similarity 0.83)
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.83)
• AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data (NeurIPS 2024 · similarity 0.83)
• CODE REPRESENTATION LEARNING AT SCALE (ICLR 2024 · similarity 0.83)