AI Paper Atlas

Research map / Image generation / Fast diffusion sampling

A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence

Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, Ming-Hsuan Yang

NeurIPS 2023 · Image generation › Fast diffusion sampling · approach: Diffusion models as perception tools · 49 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Fuse independently L2-normalized Stable Diffusion and DINOv2 featuresDINOv2-ViT-B/14 correspondence · SPair-71k test, zero-shotPCK@0.10: 55.6 to 64.0; +8.4pImproved
Fuse independently L2-normalized Stable Diffusion and DINOv2 featuresDINOv2-ViT-B/14 correspondence · SPair-71k test, supervised bottleneckPCK@0.10: 59.9 to 74.6; +14.7pImproved
Fuse independently L2-normalized Stable Diffusion and DINOv2 featuresDINOv2-ViT-B/14 correspondence · PF-Pascal, zero-shotPCK@0.10: 77.3 to 86.1Improved
Fuse independently L2-normalized Stable Diffusion and DINOv2 featuresDINOv2-ViT-B/14 correspondence · TSS, zero-shotPCK@0.05 average: 72.0 to 79.7; +7.7pImproved
Use Stable Diffusion features instead of DINOv2 featuresDense correspondence flow · TSSflow smoothness (first-order difference), average: 10.15 -> 5.90Improved
Fuse Stable Diffusion and DINOv2 featuresInstance swapping · 800-pair subsetQuality score: 63.18 -> 64.84Improved
Fuse Stable Diffusion and DINOv2 featuresInstance swapping · 800-pair subsetCLIP score: 72.63 -> 73.21Improved
Fuse Stable Diffusion and DINOv2 featuresInstance swapping · 800-pair subsetFID score (CLIP-based): 12.49 -> 12.47Improved
Use only the decoder feature rather than combining it with the encoder feature (ablation)Stable Diffusion feature extraction · Semantic correspondencecorrespondence performanceImproved
Use the decoder feature rather than a single convolution, self-attention, or cross-attention sub-layer feature (ablation)Stable Diffusion feature extraction · Semantic correspondencecorrespondence performanceImproved
Use DINOv2 token features from layer 11 instead of DINOv1 key features from earlier layersDINO correspondence features · SPair-71k subsets, 20 pairs per categoryPCK@0.10: 31.4 -> 52.7Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy