Research map / Large language models / Agents, code and math
RedSage: A Cybersecurity Generalist LLM
Naufal Suryanto, Muzammal Naseer, Pengfei Li, Syed Talal Wasim, Jinhui Yi, Juergen Gall, Paolo Ceravolo, Ernesto Damiani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| continual pretraining on cybersecurity corpora (CFW then Seed) | Qwen3-8B-Base · Cybersecurity benchmarks mean (5-shot) | mean accuracy (%): 80.81 -> 84.56 | Improved |
| add CyberFineWeb CPT only (ablation) | Qwen3-8B-Base · Cybersecurity benchmarks mean | mean accuracy (%): 80.81 -> 82.66 | Improved |
| add RedSage-Seed CPT only (ablation) | Qwen3-8B-Base · Cybersecurity benchmarks mean | mean accuracy (%): 80.81 -> 84.45 | Improved |
| combine CFW then Seed CPT instead of Seed only (ablation) | RedSage-8B · Cybersecurity benchmarks mean | mean accuracy (%): 84.45 -> 84.56 | No clear effect |
| cybersecurity continual pretraining | Qwen3-8B-Base · General benchmarks mean | mean accuracy (%): 70.86 -> 69.23 | Hurt |
| SFT with RedSage-Conv and SmolTalk2 (Ins vs Qwen3-8B) | Qwen3-8B · Cybersecurity benchmarks mean (0-shot) | mean accuracy (%): 75.71 -> 81.30 | Improved |
| add DPO after SFT (ablation) | RedSage-8B-Ins · Cybersecurity benchmarks mean (0-shot) | mean accuracy (%): 81.30 -> 81.10 | No clear effect |
| add DPO after SFT (ablation) | RedSage-8B-Ins · General benchmarks mean | mean accuracy (%): 73.34 -> 74.33 | Improved |
| add DPO after SFT (ablation) | RedSage-8B-Ins · IFEval | accuracy (%): 79.97 -> 83.44 | Improved |
| add DPO after SFT (ablation) | RedSage-8B-Ins · CTI-Bench RCM | accuracy (%): 76.70 -> 70.60 | Hurt |
| instruction tuning with RedSage data vs Qwen3-8B | Qwen3-8B · General benchmarks mean | mean accuracy (%): 65.92 -> 74.33 | Improved |
Closest papers by meaning
- Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents (ICML 2026 · similarity 0.86)
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models (ICLR 2025 · similarity 0.86)
- Cyber-Zero: Training Cybersecurity Agents without Runtime (ICLR 2026 · similarity 0.86)
- KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards (NeurIPS 2026 · similarity 0.86)
- NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security (NeurIPS 2024 · similarity 0.86)
- CyberPal.AI: Empowering LLMs with Expert-Driven Cybersecurity Instructions (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
