Research map / Image generation / GANs

Discrete Distribution Networks

ICLR 2025 · Image generation › GANs · approach: Generative adversarial network theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Discrete Distribution Networks (DDN), a generative model that outputs K discrete samples per hierarchical layer, selects the one closest to ground truth as conditioning for the next layer, and trains with a Split-and-Prune node-allocation algorithm, enabling gradient-free zero-shot conditional generation.

Problem

Generative models must fit complex high-dimensional data distributions efficiently, and most conditional generation requires per-condition training or gradient-based guidance from discriminative models; DDN targets a simpler generative form that supports zero-shot conditioning in non-pixel domains.

Approach

Approximate the data distribution with a hierarchy of discrete sample sets: each layer generates K candidate images, only the candidate closest to ground truth receives loss and feeds the next layer, so the K^L leaf outputs form an exponentially large discrete representation; Split-and-Prune rebalances node usage, and zero-shot guidance replaces the L2 selector with any black-box classifier or CLI

Key results

• DDN achieves FID 52.0 on unconditional CIFAR-10, beating VAE (106.7), VQ-VAE (117.4), Gated PixelCNN (65.9) and GLOW (46.0) only short of DCGAN (37.1) and IGEBM (38.0) in the reported table.
• On FFHQ-64x64 the default K=512, L=128 DDN reaches FID 43.1, and on CelebA-HQ-64x64 it reaches FID 35.4 versus 39 for IGN.
• Ablations on FFHQ-64x64 show removing Chain Dropout raises FID from 43.1 to 182.3, while removing Split-and-Prune raises it to 55.3.

Measured effects reported in the paper

Surprising findings

• Without Split-and-Prune the ablation FID (55.3) is not as poor as expected, which the authors attribute to Chain Dropout forcing dead nodes to receive gradients.
• The Taiji-DDN variant with K=2 gives FID 66.5 on FFHQ, showing a binary-tree latent is viable but weaker for generation.

Limitations

• The K^L output space is insufficient for complex distributions, and the authors propose patch-wise selection as a future fix.
• Split-and-Prune discards trained parameters, which the authors flag as problematic at scale.
• Pixel L2 loss and high compression blur high-frequency detail, and ZSCG needs L x K guided-model forward passes.

Experimental setup

Datasets: CIFAR-10 FFHQ CelebA-HQ MNIST

Benchmarks: CIFAR-10 FFHQ-64x64 CelebA-HQ-64x64 MNIST

Metrics: FID reconstruction FID validation accuracy KL divergence

Baselines: DCGAN IGEBM VAE VQ-VAE Gated PixelCNN GLOW IGN

Builds on: EDM VQ-VAE VQ-VAE-2 ResNet

Closest papers by meaning

• A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models (ICML 2025 · similarity 0.85)
• Compositional Discrete Latent Code for High Fidelity, Productive Diffusion Models (NeurIPS 2025 · similarity 0.85)
• Generative Conditional Distributions by Neural (Entropic) Optimal Transport (ICML 2024 · similarity 0.85)
• DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents (ICML 2024 · similarity 0.83)
• Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.83)
• U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models (ICLR 2025 · similarity 0.83)