Research map / 3D vision / 3D generation
MuTri: Multi-view Tri-alignment for OCT to OCTA 3D Image Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MuTri, a two-stage VQ-VAE framework that translates 3D OCT volumes into 3D OCTA volumes in discrete space, guided by contrastive semantic alignment and vessel structure alignment, and releases the OCTA2024 dataset.
Problem
OCT-to-OCTA 3D translation learns a mapping in continuous, infinite space with only single-view (OCTA projection map) guidance, and vanilla VQ-VAEs suffer from low codebook utilization in this large-domain-gap setting.
Approach
Pre-train OCT and OCTA VQ-VAEs for reconstruction, then train a translation VQ-VAE with a contrastive mutual-information alignment to both pre-trained 3D models (OCT and OCTA views) and a patch-level vessel structure alignment to the OCTA projection-map view, improving codebook utilization.
Key results
- On OCTA-3M, MuTri reaches 0.0707 MAE, 34.10 dB PSNR and 89.86% SSIM, versus TransPro's 0.0782, 32.56 dB and 88.22%.
- On OCTA2024 volumes, MuTri reaches 43.38 dB PSNR and 79.66% SSIM, outperforming TransPro (42.69 dB, 72.52%).
- Ablation on OCTA-6M: contrastive-inspired semantic alignment and vessel structure alignment together raise PSNR from 32.31 dB (CSA only) to 33.08 dB, and SSIM to 90.04%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MuTri instead of the strongest baseline (TransPro; Palette for SSIM) | OCT-to-OCTA translation · OCTA-3M | MAE: 0.0782 -> 0.0707 | Helped |
| Use MuTri instead of the strongest baseline | OCT-to-OCTA translation · OCTA-3M | PSNR: 32.56 -> 34.10 | Helped |
| Use MuTri instead of the strongest baseline | OCT-to-OCTA translation · OCTA-3M | SSIM: 88.24 -> 89.86 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA-6M | MAE: 0.0854 -> 0.0741 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA-6M | PSNR: 30.53 -> 33.08 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA-6M | SSIM: 88.35 -> 90.04 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA volume | MAE: 0.1098 -> 0.0828 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA volume | PSNR: 42.69 -> 43.38 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA volume | SSIM: 72.52 -> 79.66 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA projection map | MAE: 0.01056 -> 0.00870 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA projection map | PSNR: 37.85 -> 39.65 | Helped |
| Use MuTri instead of TransPro | OCT-to-OCTA translation · OCTA2024 OCTA projection map | SSIM: 87.61 -> 90.31 | Helped |
Surprising findings
- Vanilla VQ-VAE translation has low codebook utilization in OCT-to-OCTA translation, which the proposed alignments substantially improve.
Limitations
- Relies on the public OCTA-500 subsets for two of the three benchmarks, and the new OCTA2024 dataset is from a single fixed field of view.
- Diagnosis analysis is qualitative, with some ambiguity in fine vessel details noted by the authors.
Experimental setup
Datasets: OCTA-3M OCTA-6M OCTA2024
Benchmarks: OCTA-3M OCTA-6M OCTA2024
Metrics: MAE PSNR SSIM
Baselines: Pix2Pix2D Pix2Pix3D 9B18CN UNet Adjacent GAN VQ-I2I Palette TransPro
Builds on: VQ-VAE InfoNCE TransPro CUT
Closest papers by meaning
- Blood Flow Speed Estimation with Optical Coherence Tomography Angiography Images (CVPR 2025 · similarity 0.86)
- VasTSD: Learning 3D Vascular Tree-state Space Diffusion Model for Angiography Synthesis (CVPR 2025 · similarity 0.82)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.81)
- Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery (NeurIPS 2025 · similarity 0.81)
- vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation (CVPR 2025 · similarity 0.80)
- GaraMoSt: Parallel Multi-Granularity Motion and Structural Modeling for Efficient Multi-Frame Interpolation in DSA Images (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.