Research map / Image generation / Text-to-image diffusion
PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PolyJuice, a black-box, image-agnostic red-teaming method that finds a universal realness-shift direction in T2I latent space using only hard labels from a synthetic image detector, then steers generation to fool the detector.
Problem
Red teaming synthetic image detectors (SIDs) typically requires white-box access and expensive per-image optimization, which is infeasible for proprietary detectors accessed only via APIs.
Approach
Use supervised PCA with HSIC on T2I latents labeled by the black-box SID's hard predictions to find a universal steering direction that shifts latents toward the SID's failure region; apply it during sampling and transfer directions from low to high resolution by interpolation.
Key results
- PolyJuice-steered SD3.5 raises UFD success rate from 12.8% to 80.6% and RINE from 15.3% to 99.7% at 256x256.
- Calibrating SIDs on PolyJuice attacks reduces false negative rates by up to 30 points (e.g., RINE on FLUX.1-dev at 256: 52.0% to 21.8%).
- Directions computed at 256x256 transfer to 512x512, with RINE success on SD3.5 rising from 26.7% unsteered to 99.6% transferred.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply universal latent-space steering | T2I generators against UFD · COCO validation set, all models and resolutions | success rate (%): 59.6 ± 21.8 -> 89.4 ± 6.8 | Helped |
| Apply universal latent-space steering | T2I generators against RINE · COCO validation set, all models and resolutions | success rate (%): 53.7 ± 21.1 -> 91.7 ± 9.0 | Helped |
| Apply universal latent-space steering | T2I generators against T2I-specific UFD · COCO validation set, 256×256 | success rate (%): 18.3 -> 90.8 | Helped |
| Apply universal latent-space steering | T2I generators against T2I-specific RINE · COCO validation set, 256×256 | success rate (%): 8.0 -> 64.8 | Helped |
| Calibrate with PolyJuice-generated attacks | UFD · Synthetic COCO validation images, 256×256 | false negative rate (%): 69.2 -> 47.0 | Helped |
| Calibrate with PolyJuice-generated attacks | RINE · Synthetic COCO validation images, 256×256 | false negative rate (%): 52.0 -> 21.8 | Helped |
| Transfer 256×256 steering directions to 512×512 by interpolation | PolyJuice against RINE · COCO validation set, 512×512 | success rate (%): 77.6 -> 99.6 | Helped |
| Apply universal latent-space steering | FLUX.1-schnell · COCO images | FID: 17.65 -> 17.23 | Helped |
| Apply universal latent-space steering | FLUX.1-dev against RINE · COCO validation set | success rate (%): 52.4 -> 81.2 | Helped |
| Apply universal latent-space steering | SD3.5 against RINE · COCO validation set | success rate (%): 15.3 -> 99.7 | Helped |
Surprising findings
- Transferred low-resolution directions achieve success rates comparable to or better than original high-resolution directions.
- Successful attacks on RINE tend to be darker images, while those against UFD tend to have warm colors.
Limitations
- Relies on a linear dependency between SID-predicted realness and T2I latents; nonlinear dependencies are not captured.
- Requires a limited hyperparameter search for steering strength lambda_t and interval [a,b].
- Evaluated on a limited set of SIDs (UFD and RINE primarily) and three T2I models.
Experimental setup
Datasets: COCO PartiPrompts
Benchmarks: COCO validation captions PartiPrompts
Metrics: success rate (false negative rate) FID cFID precision recall density coverage CLIP-IQA CLIP score
Baselines: No Steering (unsteered T2I) DiffPGD transferred attack AdvDiff DiffAttack
Builds on: supervised principal component analysis Hilbert-Schmidt Independence Criterion DiffPGD classifier guidance
Closest papers by meaning
- RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models (NeurIPS 2026 · similarity 0.83)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.83)
- ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024 · similarity 0.81)
- T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation (CVPR 2025 · similarity 0.81)
- Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling (NeurIPS 2025 · similarity 0.81)
- Toward effective protection against diffusion-based mimicry through score distillation (ICLR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.