Research map / Image generation / Autoregressive and latent generation
WMAdapter: Adding WaterMark Control to Latent Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WMAdapter, a lightweight contextual adapter plugged into a latent diffusion VAE decoder that imprints user-specified 48-bit watermarks on the fly, trained against a frozen pretrained HiDDeN decoder with a hybrid finetuning stage for image quality.
Problem
Diffusion-generated images need copyright watermarks that scale to millions of keys without per-key finetuning, while preserving generation quality; prior diffusion-native methods either fine-tune the VAE decoder (degrading quality) or use heavy/long training pipelines.
Approach
A contextual adapter whose Fusers take both VAE decoder features and watermark bits (rather than bits alone) and add feature residuals inside the frozen VAE decoder, trained by transferring from a pretrained post-hoc HiDDeN decoder; plus a hybrid finetuning that finetunes adapter and a copy of the VAE decoder but infers with the original VAE decoder.
Key results
- WMAdapter-I reaches 34.8 dB PSNR, 0.96 SSIM and 2.5 FID versus Stable Signature's 29.7 dB, 0.87 and 3.2, at about 0.01-0.02 lower bit accuracy.
- WMAdapter-F achieves 0.99 clean bit accuracy and 0.93 under combined attacks, comparable to Stable Signature's 0.99 and 0.94.
- The adapter has only 1.3M parameters and converges in 1-2 epochs (about 5 hours on 8 A5000 GPUs) versus hundreds of epochs for post-hoc HiDDeN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the contextual adapter with image features instead of a non-contextual bit-only adapter (ablation) | WMAdapter · COCO 2017 val, clean images | bit accuracy: 0.97 -> 0.99 | Helped |
| Use the contextual adapter with image features instead of a non-contextual bit-only adapter (ablation) | WMAdapter · COCO 2017 val | PSNR: 28.7 -> 32.8 | Helped |
| Replace 1x1 fusion kernels with 3x3 convolution kernels (ablation) | WMAdapter · COCO 2017 val | bit accuracy: 0.99 -> 0.49 | Hurt |
| Replace 1x1 fusion kernels with 3x3 convolution kernels (ablation) | WMAdapter · COCO 2017 val | PSNR: 32.8 -> 12.0 | Hurt |
| Add fixed-VAE finetuning for 2,000 steps (ablation) | WMAdapter · COCO 2017 val | PSNR: 32.8 -> 33.0 | Helped |
| Add hybrid finetuning and retain the original VAE for inference (ablation) | WMAdapter · COCO 2017 val | PSNR: 32.8 -> 34.8 | Helped |
| Add hybrid finetuning and retain the original VAE for inference (ablation) | WMAdapter · COCO 2017 val, combined attacks | bit accuracy: 0.93 -> 0.91 | Hurt |
| Jointly finetune the Adapter and VAE and use the finetuned VAE at inference (ablation) | WMAdapter · COCO 2017 val, combined attacks | bit accuracy: 0.93 -> 0.94 | Helped |
| Jointly finetune the Adapter and VAE and use the finetuned VAE at inference (ablation) | WMAdapter · COCO 2017 val | PSNR: 32.8 -> 29.9 | Hurt |
| Use hybrid finetuning instead of joint finetuning (ablation) | WMAdapter · COCO 2017 val | PSNR: 29.9 -> 34.8 | Helped |
| Use WMAdapter-I instead of Stable Signature | Stable Diffusion 2.1 · COCO 2017 val | PSNR: 29.7 -> 34.8 | Helped |
| Use WMAdapter-I instead of Stable Signature | Stable Diffusion 2.1 · COCO 2017 val | FID: 3.2 -> 2.5 | Helped |
Surprising findings
- A non-contextual adapter (bits only) drops PSNR by 4.1 dB to 28.7 dB, showing image-feature conditioning matters for invisibility.
- Stable Signature-style VAE finetuning produces green/yellow spot-like artifacts, and the Joint Finetuning variant yields blurrier images with lens-flare artifacts.
Limitations
- Adapter-F produces artifacts on images with homogeneous background colors.
- Robustness to strong Gaussian noise is limited because the pretrained HiDDeN decoder was not trained for noise.
- Robustness to auto-encoder compression requires strong compression, causing about 5 dB PSNR loss.
Experimental setup
Datasets: MS-COCO 2017
Benchmarks: MS-COCO 2017 val
Metrics: bit accuracy PSNR SSIM FID tracing accuracy
Baselines: DwtDct DwtDctSvd SSL HiDDeN Stable Signature Tree-Ring WADIFF AquaLoRA RoSteALS WOUAF FSW
Builds on: Stable Signature HiDDeN Stable Diffusion
Closest papers by meaning
- Shallow Diffuse: Robust and Invisible Watermarking through Low-Dim Subspaces in Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Harnessing Frequency Spectrum Insights for Image Copyright Protection Against Diffusion Models (CVPR 2025 · similarity 0.90)
- AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRA (ICML 2024 · similarity 0.89)
- SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models (CVPR 2025 · similarity 0.89)
- StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Image Watermarks are Removable using Controllable Regeneration from Clean Noise (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.