Research map / Large language models / Language, safety and interpretability
CTX-Coder: Cross-Attention Architectures Empower LLMs for Long-Context Vulnerability Detection
Jujie Wang, Kangfeng Zheng, Bin Wu, Chunhua Wu, Yulin Yao, Jiaqi Gao, Minjiao Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-attention for contextual function embeddings (ablation) | CTX-Coder (Llama 3.1) · CTX-Vul (Total) | F1: 0.5791 -> 0.7810 | Improved |
| insert cross-attention every 4th block instead of every 8th (ablation) | CTX-Coder · CTX-Vul (Total) | F1: 0.6891 -> 0.7810 | Improved |
| insert cross-attention every 2nd block instead of every 4th (ablation) | CTX-Coder · CTX-Vul (Total) | F1: 0.7810 -> 0.8090 | Improved |
| insert cross-attention every 4th block instead of a single middle layer (ablation) | CTX-Coder · CTX-Vul (Total) | F1: 0.6210 -> 0.7810 | Improved |
| add unimodal pretraining (ablation) | CTX-Coder · CTX-Vul (Total) | F1: 0.4269 -> 0.7515 | Improved |
| add multimodal pretraining after unimodal (ablation) | CTX-Coder · CTX-Vul (Total) | F1: 0.7515 -> 0.7810 | Improved |
| use CTX-Coder instead of fully finetuned Qwen 2.5 (14B) | CTX-Coder · CTX-Vul (Medium) | F1: 0.5833 -> 0.8017 | Improved |
| use finetuned LLMs instead of non-trainable methods (LOVA, Grace) | vulnerability detection · CTX-Vul (Total) | F1: 0.1542 -> 0.5791 | Improved |
| use CTX-Coder instead of larger Qwen 2.5 (14B) on docstring generation | CTX-Coder · Python docstring generation | BLEU: 20.38 vs 20.65 | No clear effect |
| use CTX-Coder instead of larger Qwen 2.5 (14B) on cross-file completion | CTX-Coder · CrossCodeEval Python | edit similarity: 74.80 -> 71.47 | Hurt |
Closest papers by meaning
- ProSec: Fortifying Code LLMs with Proactive Security Alignment (ICML 2025 · similarity 0.87)
- CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion (NeurIPS 2026 · similarity 0.87)
- DyCodeEval: Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination (ICML 2025 · similarity 0.87)
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.86)
- Local Sparsity Enables Unsupervised LLM Safety Detection (NeurIPS 2026 · similarity 0.86)
- SecCodePRM: A Process Reward Model for Code Security (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
