Research map / Image generation / Fast diffusion sampling
Optical Diffusion Models for Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an Optical Denoising Unit: passive, phase-modulated diffractive layers implement DDPM noise prediction optically, trained with an online digital-twin algorithm, reaching FID ~131-227 on MNIST, Fashion-MNIST and Quick, Draw! at 20x20.
Problem
Diffusion model sampling requires many repeated neural network evaluations, causing high latency and energy consumption on GPUs; the paper asks whether passive optical wave propagation can perform the denoising steps.
Approach
Implement each DDPM denoising step as free-space light propagation through a fixed, passive, phase-only spatial light modulator stack, with time handled by training separate modulation-layer sets for timestep subsets (removing time embeddings), and train the layers with an online learning scheme that refines a differentiable digital twin of the experiment during training.
Key results
- Final T=1000 outputs have FID 206.6 (MNIST), 227.7 (Fashion-MNIST) and 131.4 (Quick, Draw!) for 20x20 images.
- Optical diffusion outperforms fully connected and U-Net digital denoisers across output resolutions, with ODU at 0.23 J/image versus 1.74 J (fully connected) and 5.37 J (U-Net) on an L4 GPU.
- Generation performance follows a power-law with parameter count, with fitted exponent -0.15 close to the -0.16 reported for large digital image models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ODU instead of the Fully Connected denoiser | Image generation on MNIST digits · MNIST digits | Energy/image [J]: 1.74 -> 0.23 | Helped |
| Use ODU instead of the Convolutional U-Net denoiser | Image generation on MNIST digits · MNIST digits | Energy/image [J]: 5.37 -> 0.23 | Helped |
| Use ODU instead of the Convolutional U-Net denoiser | Image generation on MNIST digits · MNIST digits | Images/s: 13.9 -> 23.0 | Helped |
| Increase the number of modulation-layer/timestep sets up to M=10 (ablation) | ODU · Image generation | generation quality | Helped |
| Increase the number of modulation-layer/timestep sets beyond M=10 (ablation) | ODU · Image generation | generation quality | Hurt |
| Increase layer resolution and layer count (ablation) | ODU · Image generation | generation quality | Helped |
| Use one modulation layer rather than multiple modulation layers (ablation) | ODU · Image generation | FID | Hurt |
| Refine the digital twin online during training | ODU experimental training · Misaligned optical experiment | MSE | Helped |
| Use experimental error backpropagation without digital-twin refinement (ablation) | ODU experimental training · Misaligned optical experiment | MSE | No clear effect |
Surprising findings
- A single modulation layer (no multi-layer benefit) is the outlier in the scaling trend.
- Backpropagating the experimental loss directly without a digital twin does not significantly decrease MSE.
Limitations
- Image resolution limited to 20x20 in main experiments; generation quality (FID > 130) is modest.
- Energy and speed advantages depend on optoelectronic input/output devices; the laboratory prototype is not optimized.
- Experimental discrepancies require the online digital-twin procedure; direct experimental backprop does not converge well.
Experimental setup
Datasets: MNIST Fashion-MNIST Quick, Draw! AFHQ
Benchmarks: MNIST Fashion-MNIST Quick, Draw! AFHQ cat
Metrics: FID Inception Score KID MSE
Baselines: fully connected denoising network convolutional U-Net denoiser offline training experimental error backpropagation
Builds on: DDPM Fresnel diffraction Split-step Fourier propagation
Closest papers by meaning
- Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network (NeurIPS 2026 · similarity 0.87)
- Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.87)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.86)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.86)
- LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling (NeurIPS 2026 · similarity 0.86)
- Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.