Research map / Adversarial robustness and security / Attacks on text and graphs
QUT-DV25: A Dataset for Dynamic Analysis of Next-Gen Software Supply Chain Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces QUT-DV25, a dynamic (eBPF-based install- and post-install-time) dataset of 14,271 PyPI packages (7,127 malicious), and benchmarks four ML and two DL detectors on it against metadata and static baselines.
Problem
Existing PyPI malicious package datasets rely on metadata or static code analysis and miss install-time and post-install-time behaviors of next-gen software supply chain attacks.
Approach
Execute each package in an isolated Raspberry Pi sandbox and record eBPF kernel and user-level traces over install plus a 120s post-install window, turning them into 36 behavioral features for malicious package classification.
Key results
- Random Forest on combined eBPF traces reached 95.99% accuracy and 96.02% F1, versus 84.44% for the metadata baseline and 95.14% for the static baseline.
- The RF model flagged 11 of 1,000 recent PyPI packages labeled benign; four confirmed malicious packages were reported to PyPI and removed.
- A CNN reached 95.28% test accuracy, comparable to ML models but with far higher training time (about 5155 s versus under 50 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine heterogeneous dynamic trace features instead of using one trace subset | RF on QUT-DV25 · QUT-DV25 test | accuracy: 94.62 -> 95.99 | Helped |
| Use PatternTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 94.62 | Hurt |
| Use OpensnoopTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 93.55 | Hurt |
| Use FiletopTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 92.01 | Hurt |
| Use SysCallTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 88.51 | Hurt |
| Use TCPTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 83.74 | Hurt |
| Use InstallTraces alone instead of CombinedTraces (ablation) | RF on QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 69.45 | Hurt |
| Use QUT-DV25 dynamic traces rather than metadata features | RF · Malicious PyPI package detection test | accuracy: 84.44 -> 95.99 | Helped |
| Use QUT-DV25 dynamic traces rather than static features | RF · Malicious PyPI package detection test | accuracy: 95.14 -> 95.99 | Helped |
| Use QUT-DV25 dynamic traces rather than metadata features | RF · Malicious PyPI package detection test | F1: 84.81 -> 96.02 | Helped |
| Use QUT-DV25 dynamic traces rather than static features | RF · Malicious PyPI package detection test | F1: 95.24 -> 96.02 | Helped |
| Train CNN rather than RF on CombinedTraces | QUT-DV25 · QUT-DV25 test | accuracy: 95.99 -> 95.28 | No clear effect |
Surprising findings
- The InstallTraces feature group alone was weak (69.45% accuracy with RF), while PatternTraces alone reached 94.62%.
- Deep learning models (CNN, Transformer) did not outperform classical ML models despite much higher compute cost.
Limitations
- Linux-only, isolated testbed; environment-sensitive attacks (Windows/macOS, user interaction, command-line arguments) may be missed.
- Post-install observation is limited to a 120 s window, so delayed or dormant payloads may be missed.
- No error bars or statistical significance tests are reported.
Experimental setup
Datasets: QUT-DV25 traces from 14,271 PyPI packages VirusTotal, NDV, Snyk labels for validation
Benchmarks: QUT-DV25
Metrics: accuracy precision recall F1-score TPR TNR FPR FNR
Baselines: MetadataDataset (Halder et al.) StaticDataset (Samaana et al.) CNN Transformer DT SVM GB
Builds on: eBPF bpftrace bcc-tools
Closest papers by meaning
- Position: Machine Learning Models Have a Supply Chain Problem (ICML 2025 · similarity 0.84)
- A Large-Scale Multi-Source Dataset Linking Hacker Community Discourse to the CVE Vulnerability Lifecycle (NeurIPS 2026 · similarity 0.84)
- GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines (NeurIPS 2026 · similarity 0.83)
- Assemblage: Automatic Binary Dataset Construction for Machine Learning (NeurIPS 2024 · similarity 0.82)
- MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents (NeurIPS 2026 · similarity 0.81)
- IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities (ICLR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.