Research map / Image generation / Autoregressive and latent generation
TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TCAM-Diff, a decoder-only triplane autoencoder for dense 3D medical volumes plus a triplane-aware cross-attention latent diffusion model, evaluated on BraTS, Pancreas and Colon CT against VAE-GAN-LDM and VQ-GAN-LDM.
Problem
Generating high-resolution 3D medical volumes (CT/MRI) with latent diffusion is memory-intensive, and existing autoencoder-based latent diffusion models lose detail and run out of memory at high resolution.
Approach
Replace the encoder with per-object learnable triplane feature planes optimized through a shared MLP decoder (decoder-only autoencoder with iterative backpropagation), and model the three planes with a diffusion network whose cross-attention layers share information across planes.
Key results
- Autoencoder reconstruction on BraTS 128^3: MSE 0.0008 and 3D SSIM 0.9311 versus VAE-GAN 0.0015 / 0.8720 and VQ-GAN 0.0079 / 0.8393.
- Generative Wasserstein distance (W-GAN critic) on BraTS: 36 vs 120 for VAE-GAN-LDM and 16 vs 160 for VQ-GAN-LDM.
- Reduced-channel triplane model reaches comparable reconstruction to VAE-GAN using 48% of its latent parameters (MSE 0.0014, SSIM 0.8985).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the decoder-only triplane autoencoder instead of VAE-GAN* | BraTS reconstruction · BraTS | MSE: 0.0014 -> 0.0008 | Helped |
| use the decoder-only triplane autoencoder instead of VAE-GAN* | BraTS reconstruction · BraTS | 3D SSIM: 0.8818 -> 0.9311 | Helped |
| use TCAM-Diff instead of VAE-GAN-LDM | BraTS generation · BraTS | Wasserstein distance: 120±20 -> 36±4 | Helped |
| use TCAM-Diff instead of VQ-GAN-LDM | BraTS generation · BraTS | Wasserstein distance: 160±20 -> 16±5 | Helped |
| use the proposed reconstruction model instead of VQ-GAN | BraTS downstream segmentation, ED label · BraTS, ED | F1 Score: 0.7815±1e-2 -> 0.9500±2e-3 | Helped |
| use the proposed reconstruction model instead of VQ-GAN | BraTS downstream segmentation, NCR/NET label · BraTS, NCR/NET | F1 Score: 0.6682±2e-2 -> 0.9273±1e-2 | Helped |
| use the proposed reconstruction model instead of VQ-GAN | BraTS downstream segmentation, ET label · BraTS, ET | F1 Score: 0.7333±2e-2 -> 0.9350±2e-3 | Helped |
| reduce triplane parameter count to 221,184 (ablation) | decoder-only autoencoder · BraTS | MSE: 0.0008 -> 0.0015 | Hurt |
| reduce triplane parameter count to 221,184 (ablation) | decoder-only autoencoder · BraTS | 3D SSIM: 0.9311 -> 0.8914 | Hurt |
| reduce triplane parameter count to 245,760 (ablation) | decoder-only autoencoder · BraTS | MSE: 0.0008 -> 0.0014 | Hurt |
| reduce triplane parameter count to 245,760 (ablation) | decoder-only autoencoder · BraTS | 3D SSIM: 0.9311 -> 0.8985 | Hurt |
| reduce triplane parameter count to 294,912 (ablation) | decoder-only autoencoder · BraTS | MSE: 0.0008 -> 0.0012 | Hurt |
Surprising findings
- VAE-GAN could not be trained on the 4-channel BraTS dataset due to out-of-memory errors, while the decoder-only model could.
Limitations
- FID was not used because the Med3D feature extractor was unavailable and the dataset was judged too small for reliable FID.
- Baseline models were retrained from public code since no pretrained weights were released.
- Encoding is slow (iterative backpropagation), with the paper accepting this as a training-time cost.
Experimental setup
Datasets: BraTS Pancreas Tumour CT Colon Cancer CT
Benchmarks: BraTS Pancreas Tumour Colon Cancer
Metrics: MSE 3D SSIM Wasserstein distance (W-GAN critic) precision recall specificity accuracy F1 score
Baselines: VAE-GAN VQ-GAN VAE-GAN-LDM VQ-GAN-LDM
Builds on: latent diffusion models triplane representation VAE-GAN VQ-GAN perceptual loss
Closest papers by meaning
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.86)
- EffiDec3D: An Optimized Decoder for High-Performance and Efficient 3D Medical Image Segmentation (CVPR 2025 · similarity 0.85)
- Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding (ICML 2024 · similarity 0.84)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.84)
- WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space (ICLR 2024 · similarity 0.84)
- DiC: Rethinking Conv3x3 Designs in Diffusion Models (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.