Research map / 3D vision / 3D generation
Introducing 3D Representation for Dense Volume-to-Volume Translation via Score Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Score-Fusion, a 3D volumetric diffusion translation model that learns a 3D fusion network over perpendicular 2D diffusion models' score estimates, initialized to average them, for medical image super-resolution, modality translation, and video super-resolution.
Problem
Volume-to-volume translation in medical imaging (e.g., MRI super-resolution and modality translation) struggles to use 3D voxel-space representations because 3D diffusion models demand large compute and data, while perpendicular 2D-model ensembles lack full 3D realism.
Approach
Learn a 3D network that fuses score estimates from perpendicularly trained 2D diffusion models at each denoising step, initialized to a weighted average of the 2D scores (as in TPDM) so 3D training becomes fine-tuning, with hierarchical 2D features aligned into 3D layers.
Key results
- On BraTS 4x super-resolution, Score-Fusion-TPDM reaches 33.24 PSNR / 0.944 SSIM with MMD 13.77 versus 32.23 / 0.922 / 29.35 for TPDM.
- Score-Fusion-TPDM achieves Dice 82.69 TC / 87.85 WT / 80.94 ET on segmentation with generated FLAIR, with recovery rates of 93.71% (TC) versus 46.27% for TPDM.
- On video super-resolution, Score-Fusion-MGLD improves DOVER from 0.748 to 0.755 over MGLD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add 3D score fusion to TPDM | TPDM · BraTS super-resolution | PSNR: 32.23 -> 33.24 | Helped |
| Add 3D score fusion to TPDM | TPDM · BraTS super-resolution | MMD: 29.35 -> 13.77 | Helped |
| Add 3D score fusion to TPDM | TPDM · BraTS modality translation | PSNR: 25.35 -> 25.26 | Hurt |
| Add 3D score fusion to TPDM | TPDM · BraTS modality translation | SSIM: 0.868 -> 0.882 | Helped |
| Add 3D score fusion to TPDM | TPDM · BraTS both conditions | PSNR: 35.12 -> 36.24 | Helped |
| Add 3D score fusion to TPDM | TPDM · BraTS both conditions | FID (1e-4): 22.2 -> 5.8 | Helped |
| Add time-space plane representations through score fusion | MGLD · VideoLQ video super-resolution | DOVER: 0.748 -> 0.755 | Helped |
| Use the smaller Score-Fusion variant instead of full-size Score-Fusion (ablation) | Score-Fusion · BraTS multi-modality fusion | PSNR: 35.6 -> 35.34 | Hurt |
| Use the smaller Score-Fusion variant instead of full-size Score-Fusion (ablation) | Score-Fusion · BraTS multi-modality fusion | Training time (GPU days): 16 -> 4 | Helped |
| Use full-size Score-Fusion rather than Score-Fusion-small (ablation) | Score-Fusion · BraTS multi-modality fusion | MMD: 8.64 -> 8.82 | Hurt |
Surprising findings
- Score-Fusion-small trained for 4 GPU days of 3D training outperforms the 16-day TPDM-based variant on PSNR/MMD in multi-modality setting (Table 3).
Limitations
- Hard to train jointly end-to-end due to compute demands of managing high-capacity 2D models with the 3D model.
- Reliance on patch-wise pre-training limits tasks needing long-range spatial information such as large-area inpainting and compressed sensing MRI.
- Evaluation limited to brain MRI (BraTS, HCP) and a single video dataset; video results show only a small DOVER gain.
Experimental setup
Datasets: BraTS 2021 HCP REDS VideoLQ
Benchmarks: BraTS 2021 HCP VideoLQ
Metrics: PSNR SSIM MMD FID Dice Recovery rate DOVER
Baselines: Pix2pix U-Net Palette I2SB Palette-3D (Med-DDPM) Pix2pix-3D U-Net-3D Palette-2.5D TPDM TOSM MADM MGLD-VSR
Builds on: TPDM Palette Med-DDPM ControlNet MGLD-VSR
Closest papers by meaning
- Learning General-purpose Biomedical Volume Representations using Randomized Synthesis (ICLR 2025 · similarity 0.88)
- Knowledge Transfer Scaling Laws for 3D Medical Imaging (NeurIPS 2026 · similarity 0.85)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.85)
- One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception (AAAI 2024 · similarity 0.85)
- CycleINR: Cycle Implicit Neural Representation for Arbitrary-Scale Volumetric Super-Resolution of Medical Data (CVPR 2024 · similarity 0.85)
- LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.