Research map / Probabilistic and scientific modeling / Variational and generative sampling
Understanding Likelihood of Normalizing Flow and Image Complexity through the Lens of Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a latent-density-concentration hypothesis explaining why normalizing flows (and PixelCNN++) assign higher likelihood to low-complexity OOD images, and shows a GMM over image complexity and latent log-likelihood fixes OOD detection.
Problem
Deep generative models, especially normalizing flows, often assign higher likelihood to OOD inputs than to in-distribution data ("failure of likelihood"), and the mechanism behind this, including failures of typicality-based tests, is unclear.
Approach
Less complex images map to high-density regions of the latent space (density concentration attraction for simpleness), which raises log p(z) and, depending on architecture, the volume term; treating image complexity (JPEG2000 code length per pixel) as an independent variable with latent log-likelihood in a GMM restores OOD detection.
Key results
- Complexity-aware GMM with Glow achieves AUROC of 100% on most noise OOD sets and 83-91% on several natural-image OOD sets when CIFAR-10 is In-Dist.
- CALT and TTL fail (AUROC below 60%) on several OOD sets, e.g., TTL gets 47.86% on SVHN when CIFAR-10 is In-Dist, and CALT gets 8.12% on CIFAR-10 when SVHN is In-Dist.
- Complexity-aware PixelCNN++ raises AUROC on CIFAR-10 In-Dist from 1.86% to 91.48% for SVHN-based OOD tests, with TIN improving from 0.75% to 96.94%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use image complexity and latent log-likelihood together in a GMM instead of likelihood alone | PixelCNN++ · CIFAR-10 -> CelebA | AUROC (%): 14.37 -> 75.71 | Helped |
| Use image complexity and latent log-likelihood together in a GMM instead of likelihood alone | PixelCNN++ · SVHN -> CIFAR-10 | AUROC (%): 1.86 -> 91.48 | Helped |
| Use image complexity and latent log-likelihood together in a GMM instead of likelihood alone | PixelCNN++ · CIFAR-10 -> SVHN | AUROC (%): 83.38 -> 95.34 | Helped |
| Use image complexity and latent log-likelihood together in a GMM instead of likelihood alone | Normalizing Flows · CIFAR-10 -> SVHN | AUROC (%): 47.86 -> 91.47 | Helped |
| Use image complexity and latent log-likelihood together in a GMM instead of likelihood alone | Normalizing Flows · SVHN -> CIFAR-10 | AUROC (%): 97.59 -> 98.01 | No clear effect |
| Use ResFlow rather than Glow as the NF base model for the complexity-aware GMM | Complexity-aware NF detection · SVHN -> Tower | AUROC (%): 100 -> 57.04 | Hurt |
| Use image complexity and log-likelihood together instead of image complexity alone | OOD detection on CIFAR-10 · CIFAR-10 -> CelebA | OOD detectability | Helped |
| Use image complexity and log-likelihood together instead of log-likelihood alone | OOD detection on CIFAR-10 · CIFAR-10 -> SVHN | OOD detectability | Helped |
Surprising findings
- Likelihood of OOD inputs can be higher than In-Dist inputs because less complex images concentrate in high-density latent regions, so the likelihood of NFs is untrustworthy.
- The typicality test fails because some OOD inputs (e.g., SVHN for CIFAR-10) have latent norms coincidentally near sqrt(d).
Limitations
- Only evaluates dataset-level OOD detection (e.g., CIFAR-10 vs SVHN), not semantic OOD detection.
- Hypothesis for autoregressive models (PixelCNN++) is supported by plausible evidence but not theoretically verified.
- ResFlow underperforms on some SVHN In-Dist OOD cases (e.g., Tower below 60% AUROC).
Experimental setup
Datasets: CIFAR-10 SVHN MNIST FMNIST ImageNet CelebA TinyImageNet LSUN OMNIGLOT NotMNIST pooling noise images manipulated CIFAR-10
Benchmarks: CIFAR-10 vs SVHN/CelebA/TIN/LSUN/noise OOD SVHN vs CIFAR-10/CelebA/TIN/LSUN/noise OOD MNIST/FMNIST vs OMNIGLOT/NotMNIST ImageNet vs pooling noise images
Metrics: AUROC AUPR
Baselines: TTL CALT LRB LRG WAIC PixelCNN++ likelihood test
Builds on: Glow iResNet ResFlow PixelCNN++ typicality test in latent space
Closest papers by meaning
- A Geometric Explanation of the Likelihood OOD Detection Paradox (ICML 2024 · similarity 0.90)
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection (NeurIPS 2025 · similarity 0.88)
- Detecting Out-of-Distribution Through the Lens of Neural Collapse (CVPR 2025 · similarity 0.88)
- Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection (CVPR 2025 · similarity 0.88)
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts (NeurIPS 2026 · similarity 0.88)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.