Research map / Image generation / Fast diffusion sampling
One Diffusion to Generate Them All
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a single 2.8B flow-matching diffusion transformer from scratch that treats all conditions and targets as a sequence of noised "views", enabling many generation and understanding tasks without task-specific modules.
Problem
Diffusion models are typically trained separately for text-to-image generation or individual predictive tasks, and existing conditional methods need task-specific adapters or auxiliary losses (e.g., identity encoders).
Approach
Model conditions and target images jointly as a sequence of views with independently sampled noise levels during training; at inference any view can be clean (t=0) conditioning while others are denoised, with the task set by a text task token.
Key results
- Achieves GenEval 0.65 at 1024x1024 with 2.8B parameters, competitive with similarly sized baselines (e.g., SD3-medium 0.62) and below FLUX-dev 0.67 at 12B.
- On Google Scanned Objects 1-view novel view synthesis, OneDiffusion reaches PSNR 20.24 vs Zero123-XL 18.93 and EscherNet 1-view 20.24 tie; 3-view known poses 21.79 vs EscherNet 24.09.
- Depth estimation on NYUv2 AbsRel 6.8 (vs Marigold 6.0) and DIODE AbsRel 29.4 (vs Marigold 31.0), showing competitive performance; ID customization is weaker than PuLID on CLIP-based ID score (0.283 vs 0.654).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OneDiffusion instead of FLUX-schnell | Text-to-image generation · GenEval | GenEval score: 0.71 -> 0.65 | Hurt |
| Use OneDiffusion instead of EscherNet | Three-view generation · Google Scanned Object | PSNR: 24.09 -> 21.79 | Hurt |
| Use OneDiffusion instead of PuLID | ID customization · Unsplash-50 | ID: 0.654 -> 0.283 | Hurt |
| Use OneDiffusion instead of PuLID | ID customization · Unsplash-50 | CLIP-T: 31.23 -> 26.80 | Hurt |
| Use OneDiffusion instead of DepthAnything-2 | Monocular depth estimation · NYUv2 | AbsRel: 4.6 -> 6.8 | Hurt |
| Use OneDiffusion instead of DepthAnything-2 | Monocular depth estimation · NYUv2 | φ1: 97.7 -> 95.2 | Hurt |
| Use OneDiffusion instead of DPT | Monocular depth estimation · DIODE | AbsRel: 18.2 -> 29.4 | Hurt |
| Use OneDiffusion instead of HDN | Monocular depth estimation · DIODE | φ1: 78.0 -> 75.2 | Hurt |
| Increase the number of conditioning views from 1 to 3 with known poses | OneDiffusion · Google Scanned Object | PSNR: 19.01 -> 21.79 | Helped |
| Provide known rather than unknown poses for 2 conditioning views | OneDiffusion · Google Scanned Object | PSNR: 19.83 -> 20.22 | Helped |
| Provide known rather than unknown poses for 3 conditioning views | OneDiffusion · Google Scanned Object | PSNR: 20.64 -> 21.79 | Helped |
Surprising findings
- On Unsplash-50 ID customization, OneDiffusion's ID similarity (0.283) is far below InstantID (0.648) and PuLID (0.654), yet the authors argue it generalizes better to expression and viewpoint changes.
Limitations
- Lower ID similarity on Unsplash-50 than specialized identity methods.
- Depth estimation trails the best diffusion/specialist baselines on some metrics; the training dataset is relatively small compared to baselines.
- Multi-view evaluation limited to Google Scanned Objects and PSNR-based comparison.
Experimental setup
Datasets: PixelProse Unsplash Coyo JourneyDB DL3DV-10K Objaverse CO3D Hypersim Google Scanned Objects NYUv2 DIODE Unsplash-50
Benchmarks: GenEval Google Scanned Objects Unsplash-50 NYUv2 DIODE
Metrics: GenEval score PSNR ID similarity CLIP-T AbsRel delta1 accuracy
Baselines: LUMINA-Next PixArt-alpha SDXL PlayGroundv2.5 IF-XL SD3-medium Hunyuan-DiT DALLE3 FLUX-dev FLUX-schnell Zero123 Zero123-XL
Builds on: Lumina-Next (Next-DiT) flow matching Stable Diffusion 3 Marigold RayDiffusion
Closest papers by meaning
- AdaDiff: Adaptive Step Selection for Fast Diffusion Models (AAAI 2025 · similarity 0.88)
- Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images (AAAI 2024 · similarity 0.88)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.88)
- Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation (ICLR 2024 · similarity 0.88)
- InstanceDiffusion: Instance-level Control for Image Generation (CVPR 2024 · similarity 0.87)
- MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.