Research map / 3D vision / 3D generation
MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MicroDiffusion, which pretrains an INR to map axially projected 2D microscopy images to a coarse 3D volume and uses it as a linearly interpolated prior inside a conditional DDPM to recover depth-resolved 3D stacks.
Problem
Reconstructing depth-resolved 3D volumes from limited 2D axial projections acquired with non-diffracting beams in fast volumetric optical microscopy, where depth information is lost.
Approach
Use a pretrained implicit neural representation as a global 3D prior: linearly interpolate its output with the noisy image at each diffusion step, and condition the denoising U-Net on the closest 2D projection and positional embeddings, combining INR structural coherence with DDPM detail recovery.
Key results
- On Vasculature, MicroDiffusion reaches SSIM 0.5787, PSNR 22.35, DICE 0.7158 versus INR at 0.5032, 21.69, 0.7136.
- On Dendrite, MicroDiffusion improves DICE from 0.4589 (INR) to 0.7557 and PSNR from 25.81 to 29.74.
- Naive diffusion without the INR prior performs very poorly (Vasculature SSIM 0.0207, DICE 0.3234), and the INR prior is essential.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add INR-guided diffusion to INR reconstruction | 3D microscopy reconstruction · Dendrite | SSIM: 0.5837 -> 0.6742 | Helped |
| add INR-guided diffusion to INR reconstruction | 3D microscopy reconstruction · Vasculature | SSIM: 0.5032 -> 0.5787 | Helped |
| add INR-guided diffusion to INR reconstruction | 3D microscopy reconstruction · Neuron | SSIM: 0.4759 -> 0.4845 | Helped |
| use Gaussian-based instead of sine-cosine positional encoding (ablation) | MicroDiffusion · Vasculature | SSIM: 0.5243 -> 0.5787 | Helped |
| add conditional feature guidance rather than remove it (ablation) | MicroDiffusion · Vasculature | SSIM: 0.5122 -> 0.5787 | Helped |
| add conditional features to ResNet block outputs instead of using cross-attention (ablation) | MicroDiffusion · Vasculature | SSIM: 0.5371 -> 0.5787 | Helped |
| use neighboring-based inference rather than only the current-coordinate INR output (ablation) | MicroDiffusion · Vasculature | SSIM: 0.4315 -> 0.5787 | Helped |
| use neighboring-based inference rather than uniformly averaging neighboring INR outputs (ablation) | MicroDiffusion · Vasculature | SSIM: 0.4996 -> 0.5787 | Helped |
| interpolate the INR prior into the noisy image rather than encode it as a condition (ablation) | MicroDiffusion · Vasculature | SSIM: 0.4178 -> 0.5787 | Helped |
| pretrain and freeze INR rather than jointly train it with the denoising U-Net (ablation) | MicroDiffusion · Vasculature | SSIM: 0.5051 -> 0.5787 | Helped |
| pretrain and freeze INR rather than leave it trainable during MicroDiffusion training (ablation) | MicroDiffusion · Vasculature | SSIM: 0.5347 -> 0.5787 | Helped |
| increase the step length (ablation) | MicroDiffusion · Dendrite, vasculature, and neuron | SSIM | Hurt |
Surprising findings
- Naive DDPM without the INR prior collapses on all datasets, showing diffusion alone cannot recover the 3D structure from limited projections.
- On the Neuron dataset, MicroDiffusion's SSIM gain over INR is small (0.4845 vs 0.4759), and the vasculature PSNR/DICE gains over INR are marginal.
Limitations
- Gaussian noise in ground truth is removed by the model, which may be undesirable when noise is part of the signal
- Performance degrades as step length increases, limiting speed-up for dense samples
- Joint or trainable INR variants underperform, so the two-stage frozen-INR design is required
Experimental setup
Datasets: Dendrite (multiphoton in vivo mouse cortex) Vasculature (multiphoton in vivo mouse cortex) Neuron (multiphoton in vivo mouse cortex)
Benchmarks: Dendrite Vasculature Neuron
Metrics: SSIM PSNR DICE
Baselines: Interpolation Interpolation-cubic INR Naive Diffusion Interpolation Diffusion Magic123
Builds on: Denoising diffusion probabilistic models Classifier-free guidance NeRP Stable Diffusion
Closest papers by meaning
- Volume Tells: Dual Cycle-Consistent Diffusion for 3D Fluorescence Microscopy De-noising and Super-Resolution (CVPR 2025 · similarity 0.87)
- Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes (CVPR 2025 · similarity 0.86)
- Discrete Spatial Diffusion: Intensity-Preserving Diffusion Modeling (NeurIPS 2025 · similarity 0.84)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.84)
- V2V3D: View-to-View Denoised 3D Reconstruction for Light Field Microscopy (CVPR 2025 · similarity 0.84)
- Single Exposure Quantitative Phase Imaging with a Conventional Microscope Using Diffusion Models (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.