Research map / Image generation / Autoregressive and latent generation
Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a GFlowNet-based latent graph module that samples diverse edge-trajectory condition representations to drive any pretrained conditional latent diffusion model toward diverse plausible images.
Problem
Conditional image generation from ambiguous prompts or conditions (text, demographics) yields low diversity when relying on random seeds or prompt diversification, which fails to capture true uncertainty.
Approach
Decompose the condition embedding into M diverse latent representations by sampling M edge-addition trajectories over a learnable latent graph with a GFlowNet (detailed balance objective, reward = exp(-MSE of diffusion noise prediction)), then decode each trajectory into a condition embedding fed to a frozen LDM.
Key results
- On 3D brain MRI, the method reaches 79% sex accuracy and 13.73 age MAE in synthesis vs 68% and 21.52 for LDM, with lower FID (0.3149 vs 0.3288).
- On natural images (COCO prompts), Inception Vendi Score rises to 28.90 vs 27.85 for CADS and 25.63 for SD2-1, with IS 10.45 vs 9.93 for PAG.
- On chest X-rays, VS increases to 7.16 vs 6.44 for SD1.5 while FID improves to 1.27 vs 1.32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use latent-graph GFlowNets instead of the SD-based baselines | Stable Diffusion v2-1-base on COCO Validation set · COCO Validation set | IS: 9.93 -> 10.45 | Helped |
| Use latent-graph GFlowNets instead of the SD-based baselines | Stable Diffusion v2-1-base on COCO Validation set · COCO Validation set | Inception VS: 27.85 -> 28.90 | Helped |
| Add latent-graph GFlowNets to the MONAI LDM | MONAI LDM · 3D Brain MRIs, synthesis | FID: 0.3288 -> 0.3149 | Helped |
| Add latent-graph GFlowNets to the MONAI LDM | MONAI LDM · 3D Brain MRIs, synthesis | Sex classification accuracy: 68% -> 79% | Helped |
| Add latent-graph GFlowNets to the MONAI LDM | MONAI LDM · 3D Brain MRIs, synthesis | Age MAE: 21.52 -> 13.73 | Helped |
| Add latent-graph GFlowNets to fine-tuned SD1.5 | fine-tuned SD1.5 on Chest X-rays · Chest X-rays | FID: 1.32 -> 1.27 | Helped |
| Add latent-graph GFlowNets to fine-tuned SD1.5 | fine-tuned SD1.5 on Chest X-rays · Chest X-rays | VS: 6.44 -> 7.16 | Helped |
| Increase the number of parallel trajectories; reducing M lowers performance (ablation) | Latent GFlowNets on 3D Brain MRIs · 3D Brain MRIs, age prediction on synthesized data | Age MAE | Helped |
| Increase the number of parallel graphs to M=10 (ablation) | Latent GFlowNets on Chest X-rays · Chest X-rays | VS | Helped |
| Increase the number of graphs from M=5 to M=10 or M=15 (ablation) | Latent GFlowNets on Chest X-rays · Chest X-rays | VS | Helped |
| Use N=25 nodes rather than fewer nodes (ablation) | Latent GFlowNets on Chest X-rays · Chest X-rays | FID and VS | Hurt |
| Vary graph sparsity rho (ablation) | Latent GFlowNets on Chest X-rays · Chest X-rays | FID and VS | Mixed |
Surprising findings
- Even with M=1 trajectory the method still outperforms the LDM baseline on brain MRI, per the ablation.
- Models with larger N (25 nodes) performed worse than smaller graphs on chest X-rays.
Limitations
- Higher computational cost for training since M trajectories are updated in parallel.
- Latent graph edges are not interpretable automatically; interpretation is manual via frequent edges.
- Anatomical plausibility of counterfactual medical images not validated clinically.
Experimental setup
Datasets: Flickr30k ADNI ABCD HCP PPMI AIBL CheXpert COCO validation
Benchmarks: COCO validation (60 prompts) 3D brain MRI synthesis and counterfactual tasks CheXpert chest X-ray generation
Metrics: Inception Vendi Score Inception Score CLIP score FID sex classification accuracy age MAE
Baselines: SD2-1 SD3.5 CADS PAG DDIM LDM GAN RadEdit SD1.5
Builds on: GFlowNets detailed balance objective latent diffusion models CADS PAG
Closest papers by meaning
- DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows (CVPR 2025 · similarity 0.87)
- Geometric Generative Modeling with Noise-Conditioned Graph Networks (ICML 2025 · similarity 0.86)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.86)
- Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation (AAAI 2024 · similarity 0.86)
- A Simple Approach to Unifying Diffusion-based Conditional Generation (ICLR 2025 · similarity 0.85)
- Editing Partially Observable Networks via Graph Diffusion Models (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.