Research map / Adversarial robustness and security / Attacks on text and graphs
Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation
Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Normalized Space Optimization (optimize in normalized space instead of pixel space) (ablation) | SORT · ImageNet 5 targets frame | attack success rate (test): near zero -> 66% (Qwen) | Improved |
| add Semantic Separation Optimization (margin loss) (ablation) | SORT · ImageNet 5 targets | attack success rate (test): average ASR gain of 6% | Improved |
| add margin loss with only two targets (ablation) | SORT · ImageNet 2 targets | attack success rate (test): Qwen frame 0.93 -> 0.93 | No clear effect |
| add SORT instead of single-target image hijacking baseline (ablation) | Qwen · ImageNet 2 targets frame | attack success rate (test): 0.00 -> 0.93 | Improved |
| increase number of targets (ablation) | SAUP on Qwen, frame · ImageNet | attack success rate (test): 93% (2 targets) -> 31% (9 targets) | Hurt |
| use corner patch (1,600 pixels) instead of frame (7,056 pixels) (ablation) | SAUP · ImageNet many targets | attack success rate: Corner ASR drops to 0% on Llava and Qwen at 8 targets | Hurt |
| increase target length in words (ablation) | SAUP · ImageNet 2 targets | attack success rate (test): 83% at 10 words; 78% Qwen, 83% Intern at 15 words | No clear effect |
| train on small 50-image RIST training set | SAUP on InternVL3 · RIST AutoDriving | attack success rate: 100% train -> 61% test | Hurt |
| increase number of targets (2 to 5) with fine-grained RIST semantics | SAUP · RIST | average attack success rate (test): 72% -> 62% | Hurt |
| slightly increase perturbed area (ablation) | SAUP · ImageNet | attack success rate | Improved |
Closest papers by meaning
- Semantic Representation Attack against Aligned Large Language Models (NeurIPS 2025 · similarity 0.88)
- Pandora's Box: Towards Building Universal Attackers against Real-World Large Vision-Language Models (NeurIPS 2024 · similarity 0.88)
- Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades (NeurIPS 2026 · similarity 0.87)
- A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1 (NeurIPS 2025 · similarity 0.87)
- BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models (CVPR 2025 · similarity 0.86)
- Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
