Research map / Adversarial robustness and security / Attacks on text and graphs
GLOW: Global Layout Aware Attacks on Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GLOW, a global layout-aware attack-plan generator for object detectors that localizes victim objects with per-category GMMs and selects target labels for all objects by layout similarity to training images, then implements plans with TOG.
Problem
Adversarial attacks on object detectors usually assume a known victim object and ignore global scene layout, so their perturbations can be caught by context-consistency defenses; vague attack requests (no specified victim) are also under-studied.
Approach
Localize victims per target label using per-category Gaussian mixture models over box center/size, then generate attack plans by matching the image's full detection layout to training-scene layouts via Hungarian matching on L1+GIoU, using the matching score as a context-consistency score; the plan is then executed with TOG.
Key results
- In white-box R1, GLOW reaches about 30% average relative improvement in layout-consistency R over the best baseline across R1-5, R1-50, R1-95.
- In zero-query black-box R2-5 on COCO, GLOW's F+T is 0.17 versus 0.12 for TOG+SAME and 0.10 for Cai [7]... with the strongest gains on transfer metrics.
- Human analysis on Pascal R2-5: 78% of GLOW results were preferred over TOG, and 70% over Cai [7].
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R1-50 · COCO17val, white-box | F+R: .18 -> .22 | Helped |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R1-50 · Pascal, white-box | F+R: .22 -> .28 | Helped |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R2-5 · COCO17val, white-box | E+R: .21 -> .32 | Helped |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R2-5 · Pascal, white-box | E+R: .22 -> .24 | Helped |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R3-50 · COCO17val, white-box | C+R: .12 -> .15 | Helped |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R3-50 · Pascal, white-box | C+R: .16 -> .12 | Hurt |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R1-50 · COCO17val, zero-query black-box | F+R: .03 -> .03 | No clear effect |
| Use GLOW global layout-aware attack planning rather than the strongest reported baseline | Object detection, R1-50 · Pascal, zero-query black-box | F+R: .11 -> .12 | Helped |
Surprising findings
- TOG+SAME, which assigns one shared target label to all objects, achieves strong fooling rates under the co-occurrence-based metric, suggesting that co-occurrence checks are easy to game.
Limitations
- Multi-category requests (R3 with different labels) are left as future work; only same-label multi-object requests are handled.
- Layout prior is built from the training dataset of the victim/surrogate model, so plans depend on that dataset's layout statistics.
- Black-box transfer performance remains low in absolute terms for R3 (C+R near 0.01-0.04).
Experimental setup
Datasets: MS COCO 2017 train MS COCO 2017 val Pascal VOC
Benchmarks: COCO 2017 val Pascal VOC
Metrics: fooling rate T (layout consistency score) R (layout recall) E C human pairwise preference
Baselines: TOG TOG+RAND TOG+SAME Cai [7]
Builds on: TOG Cai et al. context-aware attack Hungarian matching GIoU
Closest papers by meaning
- UV-Attack: Physical-World Adversarial Attacks on Person Detection via Dynamic-NeRF-based UV Mapping (ICLR 2025 · similarity 0.85)
- Full-Distance Evasion of Pedestrian Detectors in the Physical World (NeurIPS 2024 · similarity 0.85)
- PAD: Patch-Agnostic Defense against Adversarial Patch Attacks (CVPR 2024 · similarity 0.84)
- Seeing is Not Believing: Adversarial Natural Object Optimization for Hard-Label 3D Scene Attacks (CVPR 2025 · similarity 0.84)
- Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach (AAAI 2025 · similarity 0.84)
- Everywhere Attack: Attacking Locally and Globally to Boost Targeted Transferability (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.