Research map / Vision-language and multimodal / Visual question answering
STING-BEE: Towards Vision-Language Model for Real-World X-ray Baggage Security Inspection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces STCray, a 46,642-scan multimodal X-ray baggage security dataset with STING-protocol captions, and STING-BEE, a LLaVA-style VLM fine-tuned for scene comprehension, referring threat localization, grounding, and VQA.
Problem
Existing X-ray baggage security datasets are uni-modal with simple labels, lack real-world threats and concealment, and vision models are closed-set; general VLMs fail on X-ray imagery.
Approach
Build a caption-rich X-ray dataset via a systematic protocol that varies threat position, orientation and occlusion, then instruction-tune a LLaVA-1.5-style model with task-identification tokens ([refer], [grounding]) to unify several X-ray vision-language tasks.
Key results
- STING-BEE reaches 34.69 F1 and 29.80 mAP on cross-domain scene comprehension, versus 18.45 F1 for MiniGPT.
- VQA overall accuracy of 52.81% versus 41.94% for LLaVA 1.5 and 36.62% for MiniGPT.
- Overall referring/grounding acc@0.25 of 21.51%, against 12.60% for Florence-2 and 11.05% for MiniGPT-v2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on STCray with multi-task instruction tuning and threat visual-grounded tuning | STING-BEE · STCray, SIXray, and PIDray VQA | accuracy (Overall): 41.94 -> 52.81 | Helped |
| Train on STCray with multi-task instruction tuning and threat visual-grounded tuning | STING-BEE · Cross-domain scene comprehension | F1: 18.45 -> 34.69 | Helped |
| Train on STCray with multi-task instruction tuning and threat visual-grounded tuning | STING-BEE · Cross-domain scene comprehension | mAP: 20.99 -> 29.80 | Helped |
| Train on STCray with multi-task instruction tuning and threat visual-grounded tuning | STING-BEE · Cross-domain visual grounding and referring localization | acc@0.5 (Overall): 5.7 -> 8.7 | Helped |
| Train on STCray with multi-task instruction tuning and threat visual-grounded tuning | STING-BEE · Cross-domain visual grounding and referring localization | acc@0.25 (Overall): 12.60 -> 21.51 | Helped |
| Train on STCray for cross-domain threat localization | STING-BEE · Combined SIXray and PIDray, common threat categories | Accuracy@0.5 (%): 4.4 -> 8.7 | Helped |
| Train on STCray for cross-domain threat localization | STING-BEE · Combined SIXray and PIDray, common threat categories | Accuracy@0.25 (%): 8.6 -> 21.5 | Helped |
Surprising findings
- GPT-4 and Gemini-1.5 Pro fail to identify threat items in X-ray scans, and LLaVA-NeXT interprets baggage scans as medical images.
Limitations
- Localization is imprecise under severe occlusion, covering only part of a threat.
- Struggles with multiple threats, including confusing similar categories (wrench vs pliers) and grouping instances.
- Absolute grounding accuracy remains low (acc@0.5 of 8.7% overall).
Experimental setup
Datasets: STCray SIXray PIDray COMPASS-XP
Benchmarks: STCray test SIXray PIDray COMPASS-XP
Metrics: F1 mAP accuracy acc@0.5 acc@0.25
Baselines: MiniGPT-v2 Florence-2 LLaVA 1.5 CLIP Long-CLIP Faster RCNN DETR PSN SDANet
Builds on: LLaVA 1.5 CLIP Vicuna
Closest papers by meaning
- Dual-view X-ray Detection: Can AI Detect Prohibited Items from Dual-view X-ray Images like Humans? (CVPR 2025 · similarity 0.81)
- CSPCL: Category Semantic Prior Contrastive Learning for Deformable DETR-Based Prohibited Item Detectors (NeurIPS 2025 · similarity 0.81)
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.80)
- MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs (ICLR 2026 · similarity 0.80)
- LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · similarity 0.80)
- HoneyBee: Data Recipes for Vision-Language Reasoners (CVPR 2026 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.