Research map / Image generation / Autoregressive and latent generation
Tokenizer-Generator Coupling in Medical Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Controlled ChestMNIST-64 factorial study showing that discrete tokenizer (VQ/LFQ/FSQ), generator family, and sampler settings interact, so they cannot be ranked independently; sampler retuning of D3PM/SEDD yields large FID gains.
Problem
Latent medical image generators usually treat the tokenizer as fixed preprocessing; it is unclear whether tokenizer, generator and sampler choices can be evaluated separately for low-resolution medical image generation.
Approach
Crosses discrete tokenizers (VQ, LFQ, FSQ at 1K/2K/4K vocabularies) with six discrete generators and continuous VAE/AE latents under a shared 8x8 latent grid, and shows that the best quantizer changes with the generator and that validation-selected sampler settings can reorder tokenizers and generators, framed as rate-distortion-modelability.
Key results
- On the seed-verified vocabulary-1024 block, 4 of 9 pairwise quantizer comparisons exceed three seed standard deviations, including an LFQ-vs-FSQ reversal from MaskGIT to D3PM.
- Validation-tuned D3PM on LFQ-1024 reduces FID-192 from 0.44 to 0.09 (100 steps), and tuned SEDD from 0.41 to 0.10.
- FID-192 ranks consistently with standard FID-2048 (Spearman 0.89) and with a label-free classifier two-sample test (0.86).
Surprising findings
- Reconstruction PSNR does not reliably predict generation quality; VAE-c8 has the best PSNR (34.4) but the worst LDM generation in the channel sweep.
- Default 1,000-step D3PM and SEDD samplers are suboptimal; 100-500 steps give 4-10x FID improvements, and tuned D3PM overtakes the AR baseline.
Limitations
- Single training seed for most grid cells; only the vocabulary-1024 block is seed-verified.
- Continuous references were not given an equivalent sampler sweep, so the tuned discrete vs default continuous comparison is asymmetric.
- Low-resolution 64x64 unconditional setting with non-clinical FID-based metrics; claims do not transfer to clinical use.
Experimental setup
Datasets: ChestMNIST PneumoniaMNIST OrganAMNIST
Benchmarks: ChestMNIST-64 test set PneumoniaMNIST OrganAMNIST
Metrics: FID-192 FID-2048 PSNR SSIM LPIPS domain-FID classifier two-sample AUC nearest-neighbour memorization ratio
Baselines: MaskGIT LDM RF BFN DFM
Builds on: MAGVIT-v2 D3PM SEDD MaskGIT rectified flow
Closest papers by meaning
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.86)
- An Image is Worth 32 Tokens for Reconstruction and Generation (NeurIPS 2024 · similarity 0.86)
- ImageFolder: Autoregressive Image Generation with Folded Tokens (ICLR 2025 · similarity 0.85)
- Image Understanding Makes for A Good Tokenizer for Image Generation (NeurIPS 2024 · similarity 0.85)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.85)
- End-to-End Training for Unified Tokenization and Latent Denoising (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.