Research map / Image generation / Super-resolution
Image Neural Field Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains latent diffusion models over image neural field latents (CLIF renderer on a LDM autoencoder) so that diffusion can generate photorealistic images at arbitrary resolution, including 2K, from mixed-resolution data.
Problem
Diffusion models typically learn fixed-resolution image distributions, requiring separate super-resolution models for high-resolution synthesis and struggling with mixed-resolution training data.
Approach
Convert a latent diffusion autoencoder into an image neural field autoencoder by adding a convolutional local image function (CLIF) renderer trained on random-scale fixed-size patches with L1, perceptual, and GAN losses, then run diffusion on the resolution-agnostic latent.
Key results
- On FFHQ at 1K/1K, INFD achieves pFID 5.57 versus 8.97 for LDM + LIIF and 16.04 for LDM + Real-ESRGAN (pFID@50K).
- On Mountains, INFD reaches pFID 7.53 at 256/1K versus 17.36 for LDM + Real-ESRGAN.
- Random downsampling during training improves 1K/1K pFID from 6.04 to 5.57 on FFHQ even when training images are all 1024 resolution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use INFD instead of LDM followed by super-resolution | high-resolution image generation on FFHQ · FFHQ | pFID 1K/1K: 16.04 -> 5.57 | Helped |
| Use INFD instead of LDM followed by Real-ESRGAN | high-resolution image generation on Mountains · Mountains | pFID 1K/1K: 10.39 -> 5.13 | Helped |
| Randomly downsample training images to provide scale-varied patch supervision (ablation) | INFD autoencoder · FFHQ | pFID 1K/1K: 6.04 -> 5.57 | Helped |
| Randomly downsample training images to provide scale-varied patch supervision (ablation) | INFD autoencoder · FFHQ | pFID 512/1K: 6.82 -> 6.64 | Helped |
| Balance training resolution by sampling from the high-resolution subset with probability 0.5 (ablation) | INFD trained on FFHQ 6K-Mix · FFHQ 6K-Mix | pFID 1K/1K: 11.3 -> 6.9 | Helped |
| Balance training resolution by sampling from the high-resolution subset with probability 0.5 (ablation) | INFD trained on FFHQ 6K-Mix · FFHQ 6K-Mix | pFID 512/1K: 10.8 -> 7.74 | Helped |
Surprising findings
- Random downsampling of ground truth during training improves quality even when the target is a fixed 1024 resolution, contrary to LIIF's observation with L1-only loss.
- Standard LDM with latent denoiser generating high resolution directly produces repetitive patterns, motivating explicit super-resolution pipelines.
Limitations
- Assumes training data are scale-consistent; violated by datasets with noisy low-res and clean high-res subsets.
- Text-to-image model only finetuned on a small high-resolution LAION subset, requiring prompts like '4k' and generalizing poorly to some categories.
- GANs still achieve better FID for single-class generation than the proposed method.
Experimental setup
Datasets: FFHQ Mountains LAION-5B subset
Benchmarks: FFHQ Mountains
Metrics: pFID FID
Baselines: LDM LDM + LIIF LDM + ITSRN LDM + LTE LDM + Real-ESRGAN AnyRes-GAN
Builds on: Latent diffusion models LIIF AnyResGAN DPS
Closest papers by meaning
- Neural Diffusion Models (ICML 2024 · similarity 0.89)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.88)
- Adversarial Diffusion Compression for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.88)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.87)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.87)
- RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.