Research map / Image generation / Fast diffusion sampling
Discrete Modeling via Boundary Conditional Diffusion Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes boundary conditional diffusion, which estimates discrete boundaries of embeddings as priors and rescales the forward ODE trajectory so continuous diffusion models match discrete data areas, improving language modeling and categorical image generation.
Problem
Continuous diffusion models treat discrete data as single points, so learned probability contours mismatch the discrete areas that decode to each token or pixel value, hurting discrete modeling.
Approach
Estimate the discrete boundary of each data point as a prior via a deterministic (ODE-based) forward trajectory crossing point t0, then rescale the forward trajectory to start from the boundary so noisy samples are conditioned on the boundary; the reverse process is adjusted accordingly.
Key results
- On IWSLT14 de-en the method reaches 33.42 BLEU versus 31.58 for the Difformer base and 34.31 for the Transformer.
- On CIFAR-10 with categorical pixels (trainable embedding) the method achieves FID 10.99 versus 19.26 for reproduced Bit Diffusion, a new state of the art in that setting.
- Ablation on IWSLT14 shows forward-only rescaling gives 33.02 BLEU and forward-plus-reverse gives 33.42 BLEU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add boundary-conditioned diffusion with forward and reverse trajectory rescaling | Difformer · IWSLT14 de-en | BLEU: 31.58 -> 33.42 | Helped |
| Add boundary-conditioned diffusion with forward and reverse trajectory rescaling | Difformer · WMT14 en-de | BLEU: 24.80 -> 26.69 | Helped |
| Add boundary-conditioned diffusion with forward and reverse trajectory rescaling | Difformer · WMT16 en-ro | BLEU: 30.08 -> 33.15 | Helped |
| Add boundary-conditioned diffusion with forward and reverse trajectory rescaling | Difformer · Gigaword | ROUGE-1: 35.47 -> 36.44 | Helped |
| Rescale the forward trajectory only (ablation) | Difformer · IWSLT14 | BLEU: 31.58 -> 33.02 | Helped |
| Rescale the forward trajectory only (ablation) | Difformer · WMT16 | BLEU: 30.08 -> 32.86 | Helped |
| Add reverse-trajectory rescaling after forward-trajectory rescaling (ablation) | Difformer with forward rescaling · IWSLT14 | BLEU: 33.02 -> 33.42 | Helped |
| Add reverse-trajectory rescaling after forward-trajectory rescaling (ablation) | Difformer with forward rescaling · WMT16 | BLEU: 32.86 -> 33.15 | Helped |
| Replace the forward trajectory with optimal transport (ablation) | Boundary-conditioned Difformer · WMT16 | BLEU: 33.15 -> 33.65 | Helped |
| Use the rescaled vector-field objective instead of the x0-prediction objective (ablation) | Boundary-conditioned Difformer · IWSLT14 de-en | BLEU: 33.42 -> 33.49 | No clear effect |
| Increase confidence factor r (ablation) | Boundary-conditioned Bit Diffusion, binary coding · CIFAR-10 | FID: 10.37 -> 7.39 -> 5.33 -> 3.86 | Helped |
| Increase confidence factor r (ablation) | Boundary-conditioned Bit Diffusion, fixed embedding · CIFAR-10 | FID: 12.96 -> 11.35 -> 10.80 -> 9.15 | Helped |
Surprising findings
- Optimal-transport variant of the forward process gives better WMT16 BLEU (33.65) than the main method (33.15).
- Replacing the exact training loss with an upper bound costs only about 0.2% BLEU while improving loss computation efficiency.
Limitations
- Designed only for continuous diffusion models; discrete diffusion models avoid the problem.
- Confidence factor r is set empirically and large values can cause collapse in unconditional generation.
- Reverse-process trajectory instability; fixing the initial path sometimes performs better.
Experimental setup
Datasets: IWSLT14 de-en WMT14 en-de WMT16 en-ro Gigaword CIFAR-10
Benchmarks: IWSLT14 de-en WMT14 en-de WMT16 en-ro Gigaword CIFAR-10
Metrics: BLEU ROUGE-1/2/L FID
Baselines: Transformer D3PM DiffuSeq SeqDiffuSeq Difformer Dinoiser SEDD Bit Diffusion tauLDR SDDM DDPM DDIM
Builds on: Difformer Bit Diffusion DDPM DDIM Flow Matching
Closest papers by meaning
- Continuous Diffusion Model for Language Modeling (NeurIPS 2025 · similarity 0.90)
- LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling (NeurIPS 2026 · similarity 0.89)
- Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.89)
- Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.89)
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising (NeurIPS 2026 · similarity 0.89)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.