AI Paper Atlas

Research map / Image generation / Fast diffusion sampling

A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting

Wouter Van Gansbeke, Bert De Brabandere

ECCV 2024 · Image generation › Fast diffusion sampling · approach: Diffusion models as perception tools · 17 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace shallow autoencoder (2M params) with finetuned Stable Diffusion VAE (84M) (ablation)LDMSeg · COCO val2017 (class-agnostic panoptic)PQ: 50.8 -> 50.9No clear effect
add KL loss with weight above 1e-5 to the autoencoder (ablation)LDMSeg autoencoder · COCO val2017reconstruction qualityHurt
use bit encoding instead of color encoding for instance IDs (ablation)LDMSeg autoencoder · COCO autoencoder reconstructionPQ: 89.1 -> 89.9Improved
use bit encoding instead of positional encoding (ablation)LDMSeg autoencoder · COCO autoencoder reconstructionPQ: 88.2 -> 89.9Improved
increase DDIM inference steps from 1 to 50 (ablation)LDMSeg · COCO class-agnostic panoptic, 50k itersPQ: 8.4 -> 51.9Improved
increase sampling steps beyond 50 (ablation)LDMSeg · COCO class-agnostic panoptic, 50k itersPQ: 51.9 -> 52.2No clear effect
swap Stable Diffusion VAE image encoder for ViT-B/14 DINOv2 weights (ablation)LDMSeg · COCO panoptic component analysisPQ: 40.6 -> 43.7Improved
use DDPM scheduler instead of DDIM (ablation)LDMSeg · COCO panoptic component analysisPQ: 43.7 -> 44.3Improved
add EMA of model weights (ablation)LDMSeg · COCO panoptic component analysisPQ: 40.3 -> 40.6Improved
train jointly on semantic, panoptic and depth with task embeddings versus single-taskLDMSeg · COCO val2017 multi-taskSemantic seg mIoU: 61.5 -> 60.1Hurt
train jointly on three tasks versus single-taskLDMSeg · COCO val2017 multi-taskPanoptic PQ: 44.3 -> 44.1No clear effect
add CLIP image embeddings or BLIP captions via cross-attention (ablation)LDMSeg · COCO class-agnosticsample qualityNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy