Research map / Image generation / Autoregressive and latent generation
Watermarking Autoregressive Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts KGW LLM watermarking to autoregressive image (and audio) token generation, improving reverse cycle-consistency via tokenizer decoder/replica-encoder finetuning and adding a localized-watermark quadrant synchronization layer to recover geometric transforms.
Problem
Autoregressive image generators have no token-level watermark; decoding then re-tokenizing generated images breaks token sequences (low reverse cycle-consistency), so watermarks are lost under transformations like JPEG, flips, and crops.
Approach
Apply KGW green/red token watermarking to image tokens, then fine-tune the VQ decoder and a replica encoder for reverse cycle-consistency with augmentations (keeping codebook and encoder fixed), plus a post-hoc localized-watermark synchronization layer that estimates and reverts flips, rotations and crops before detection.
Key results
- Token match between generated tokens and re-tokenized images is about 0.66 for Taming without transformations and drops to near 0 under rotation, flip, and crop; finetuning with augmentations raises robustness to valuemetric attacks (e.g., JPEG TPR from 0.26 to 0.92 for Taming).
- Watermarked generations keep FID at or below 16.7 (unwatermarked baseline) for finetuned variants; the synchronized variant reaches 17.3.
- The method with synchronization (FtAugSync) achieves TPR of 0.82 under geometric transformations on Taming, versus 0.01 for the base model, while post-hoc baselines like CIN and MBRS drop to about 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RCC-finetune the tokenizer (ablation) | Taming watermark · ImageNet, unmodified images | token match | Helped |
| RCC-finetune the tokenizer (ablation) | Taming watermark · Taming, low false positive rates | TPR | Helped |
| add augmentations to RCC finetuning (ablation) | Taming watermark · Taming, valuemetric transformations | TPR at 1% FPR: 0.45 -> 0.92 | Helped |
| add augmentations to RCC finetuning (ablation) | RAR-XL watermark · RAR-XL, valuemetric transformations | TPR at 1% FPR: 0.22 -> 0.98 | Helped |
| add synchronization (ablation) | augmented RCC-finetuned watermark · Taming, geometric transformations | TPR at 1% FPR: 0.01 -> 0.82 | Helped |
| add synchronization (ablation) | augmented RCC-finetuned watermark · Chameleon, geometric transformations | TPR at 1% FPR: 0.02 -> 0.64 | Helped |
| add synchronization (ablation) | augmented RCC-finetuned watermark · Taming, valuemetric transformations | TPR at 1% FPR: 0.92 -> 0.83 | Hurt |
| add synchronization (ablation) | Taming watermark · ImageNet generations | FID: 16.7 -> 17.3 | Hurt |
| watermark with RCC finetuning, without synchronization (ablation) | Taming image generator · ImageNet generations | FID | No clear effect |
| add augmentations to RCC finetuning (ablation) | Moshi watermark · Audio, valuemetric transformations | TPR at 1% FPR: 0.64 -> 0.80 | Helped |
| RCC-finetune without augmentations (ablation) | Moshi watermark · Audio, time-frequency transformations | TPR at 1% FPR: 0.24 -> 0.14 | Hurt |
| add augmentations to RCC finetuning (ablation) | Moshi watermark · Audio, time-frequency transformations | TPR at 1% FPR: 0.14 -> 0.24 | Helped |
Surprising findings
- Finetuning the tokenizer also improved robustness to neural compression and diffusion purification, not only valuemetric transformations.
- Joint watermarking of text and images improves detection, but integrating a strong noise-corrupted image signal can degrade detection.
Limitations
- Targets only VQ-based autoregressive image models; not applicable to continuous-representation or hybrid AR-diffusion models.
- Synchronization uses a fixed quadrant pattern; geometric attacks combined with removal attacks can break it, and moderate valuemetric noise can cause false transform estimates.
- Audio time-frequency robustness remains low (TPR 0.24 under time transforms).
Experimental setup
Datasets: ImageNet COCO prompts VoxPopuli Open Assistant
Benchmarks: ImageNet class-conditional generation (FID) COCO prompts (Chameleon) VoxPopuli audio
Metrics: token match TPR at 1% FPR FID PSNR PESQ MOSNet
Baselines: CIN MBRS Trustmark WAM AudioSeal
Builds on: KGW watermark VQGAN LocalMark (localized watermarking) Moshi
Closest papers by meaning
- A Locally Tokenized Generative Model for Robust Time-Series Watermarking (NeurIPS 2026 · similarity 0.89)
- BitMark: Watermarking Bitwise Autoregressive Image Generative Models (NeurIPS 2025 · similarity 0.89)
- On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.87)
- Robust Distortion-Free Watermark for Autoregressive Audio Generation Models (NeurIPS 2025 · similarity 0.87)
- Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models (NeurIPS 2024 · similarity 0.86)
- On the Robustness of Watermarking for Autoregressive Image Generation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.