Research map / Image generation / Text-to-image diffusion
PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free text-guided image-to-image method that embeds a reference image into text-described scenes to create optical illusion hidden pictures by transferring diffusion feature phase spectra between DDIM reconstruction and sampling trajectories.
Problem
Generating optical illusion hidden pictures, where a reference image is naturally blended into a text-described scene while keeping both the hidden structure and the scene semantics discernible; existing I2I and controllable T2I methods suffer from structure-semantic conflict and unnatural results.
Approach
A plug-and-play phase transfer module that injects the phase spectrum of DDIM-inverted reference features into the text-guided sampling features over the early denoising steps, with a decayed blending schedule and an asynchronous distance parameter to control hidden-content discernibility.
Key results
- PTDiffusion achieves Aesthetic Score 6.37, CLIP Score 0.31, and LPIPS 0.64, leading all compared methods (best baseline PTI: 6.18, 0.28, 0.52).
- Asynchronous phase transfer with async distance d from -9 to 9 raises DINO structure similarity from 0.880 to 0.926 while CLIP score drops only from 0.309 to 0.293.
- In a 16-participant user study, PTDiffusion outscores all baselines by a large margin on contextual naturalness and illusion balance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Phase transfer module (PTM) with asynchronous phase transfer vs. prior text-guided I2I and controllable T2I baselines | PTDiffusion on SD v1.5 · Illusion synthesis (80 reference images) | Aesthetic Score, CLIP Score, LPIPS: Aesthetic 6.37 vs best baseline 6.24; CLIP 0.31 vs 0.29; LPIPS 0.64 vs 0.56 | Helped |
| Remove refining stage (phase transfer applied over all steps) (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | CLIP Score: 0.31 -> 0.24 | Hurt |
| Remove refining stage (phase transfer applied over all steps) (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | Aesthetic Score: 6.37 -> 5.95 | Hurt |
| Remove decayed phase blending schedule (phase decay) (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | Aesthetic Score: 6.37 -> 6.24 | Hurt |
| Remove decayed phase blending schedule (phase decay) (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | CLIP Score: 0.31 -> 0.28 | Hurt |
| Use DDIM inversion (deterministic guidance trajectory) instead of forward diffusion (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | Aesthetic Score and CLIP Score: Aesthetic 6.12 -> 6.37; CLIP 0.25 -> 0.31 | Helped |
| Increase async distance d (from 0 to 9) to transfer phase from later guidance features (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | Structure Similarity: 0.908 -> 0.926 at d=9 | Helped |
| Increase async distance d (from 0 to 9) (ablation) | PTDiffusion · Illusion synthesis (80 reference images) | CLIP Score: 0.307 -> 0.293 at d=9 (mild decline) | Hurt |
Surprising findings
- Phase transfer improves results without any training, fine-tuning, or online optimization, as a plug-and-play module on SD v1.5.
Limitations
- Relies on hyperparameter tuning per case (best-effect settings for baselines; async distance manually tuned)
- Evaluation uses a small set of 80 reference images and a 16-person user study
Experimental setup
Metrics: Aesthetic Score CLIP Score LPIPS DINO-ViT self-similarity structure similarity user study ratings (contextual naturalness, illusion balance
Baselines: Null-text inversion Prompt-tuning inversion PAP FBSDiff SDEdit ControlNet
Builds on: Latent Diffusion Model DDIM inversion classifier-free guidance SDEdit
Closest papers by meaning
- Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models (CVPR 2024 · similarity 0.88)
- PS-Diffusion: Photorealistic Subject-Driven Image Editing with Disentangled Control and Attention (CVPR 2025 · similarity 0.88)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.87)
- Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation (ICLR 2024 · similarity 0.86)
- Image Inpainting via Tractable Steering of Diffusion Models (ICLR 2024 · similarity 0.86)
- Relay Diffusion: Unifying diffusion process across resolutions for image synthesis (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.