Research map / Adversarial robustness and security / Attacks on text and graphs
Fooling Polarization-Based Vision using Locally Controllable Polarizing Projection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts a commercial one-chip LCD projector by removing its front polarizer to achieve locally controllable polarizing projection, and uses it to physically fool polarization-based glass segmentation (PGSNet) and shape-from-polarization (SfP-wild) models.
Problem
Whether polarization-based vision algorithms are vulnerable to adversarial attacks, and whether such attacks can be realized physically in an imperceptible way, since human eyes cannot perceive polarization.
Approach
Modify an LCD projector by tearing off its front polarizing film so each LCD cell controls the polarization angle of projected light at constant intensity, producing uniform-looking white projections that polarization cameras can record; optimize per-grid projection patterns via softmax-weighted candidate images with an added LE loss and EOT augmentation.
Key results
- On glass segmentation in the physical world, the optimized grid-8 perturbation reduces IoU from 0.715 (unpolarized) to 0.422, versus 0.509 for random perturbation.
- Digital-world Ours-8 with the LE loss lowers IoU to 0.200 and raises MAE to 0.719, compared with 0.101 MAE for unpolarized projection.
- The attack on SfP-wild is shown qualitatively, producing a fake cube-shaped normal map in both digital and physical settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use optimized polarizing projections instead of unpolarized illumination | PGSNet glass segmentation attack · Glass segmentation, digital and physical worlds | MAE: 0.101 -> 0.719 (digital, Ours-8); 0.101 -> 0.399 (physical, Ours-16) | Helped |
| Use optimized polarizing projections instead of random perturbations | PGSNet glass segmentation attack · Glass segmentation, digital world | MAE: 0.204 -> 0.719 (grid size 8); 0.246 -> 0.698 (grid size 16); 0.329 -> 0.678 (grid size 32) | Helped |
| Use optimized polarizing projections instead of random perturbations | PGSNet glass segmentation attack · Glass segmentation, physical world | MAE: 0.167 -> 0.318 (grid size 8); 0.287 -> 0.399 (grid size 16); 0.257 -> 0.347 (grid size 32) | Helped |
| Add EOT (ablation) | PGSNet glass segmentation attack · Glass segmentation, digital world | MAE: 0.745 -> 0.719 (grid size 8); 0.735 -> 0.698 (grid size 16); 0.674 -> 0.678 (grid size 32) | Mixed |
| Add EOT (ablation) | PGSNet glass segmentation attack · Glass segmentation, physical world | MAE: 0.321 -> 0.318 (grid size 8); 0.372 -> 0.399 (grid size 16); 0.307 -> 0.347 (grid size 32) | Mixed |
| Add LE loss (ablation) | PGSNet glass segmentation attack · Glass segmentation, digital world | MAE: 0.569 -> 0.719 (grid size 8); 0.588 -> 0.698 (grid size 16); 0.680 -> 0.678 (grid size 32) | Helped |
| Add LE loss (ablation) | PGSNet glass segmentation attack · Glass segmentation, physical world | MAE: 0.271 -> 0.318 (grid size 8); 0.404 -> 0.399 (grid size 16); 0.342 -> 0.347 (grid size 32) | Helped |
| Use grid size 8 rather than grid size 32 (ablation) | Optimized PGSNet glass segmentation attack · Glass segmentation, digital world | MAE: 0.678 -> 0.719 | Helped |
| Use grid size 32 rather than grid size 8 (ablation) | Optimized PGSNet glass segmentation attack · Glass segmentation, physical world | MAE: 0.318 -> 0.347 | Mixed |
| Use polarizing projection to target an artificial shape | SfP-wild · Shape-from-polarization | normal-map error | Helped |
Surprising findings
- Physical attacks with lower-resolution grids (32) performed better in the physical world than high-resolution grids (8), while high resolution was more effective digitally.
Limitations
- Low projector luminance makes the attack less effective in bright environments.
- Whitebox attack assumes full access to the target model.
- Simplified setup with co-located projector and camera and no full light-transport simulation.
Experimental setup
Datasets: indoor polarization captures of candidate and background images (11 scenes)
Benchmarks: PGSNet glass segmentation attack (11 scenes) SfP-wild shape estimation attack
Metrics: MAE IoU
Baselines: Random perturbation Unpolarized projection
Builds on: Expectation Over Transformation PGSNet SfP-wild projector-based physical attacks
Closest papers by meaning
- SPIDeRS: Structured Polarization for Invisible Depth and Reflectance Sensing (CVPR 2024 · similarity 0.85)
- PolarFree: Polarization-based Reflection-Free Imaging (CVPR 2025 · similarity 0.84)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.84)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.84)
- ProjAttacker: A Configurable Physical Adversarial Attack for Face Recognition via Projector (CVPR 2025 · similarity 0.82)
- Learning to Remove Wrinkled Transparent Film with Polarized Prior (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.