Research map / Label-efficient and robust learning / Self-supervised and contrastive

Pose-Guided Self-Training with Two-Stage Clustering for Unsupervised Landmark Discovery

CVPR 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Pretext-task visual pre-training · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes D-ULD and D-ULD++, unsupervised landmark discovery methods built on Stable Diffusion features, adding self-training with clustering pseudo-labels, a VAE-based pose-guided proxy task, and two-stage pose-then-keypoint clustering.

Problem

Unsupervised landmark discovery (ULD) for object categories such as faces and cats, where existing methods struggle under large pose and intra-class variations.

Approach

Uses pre-trained Stable Diffusion features aggregated over layers and timesteps; a zero-shot clustering baseline is followed by self-training on clustering pseudo-labels, and a VAE pose-latent proxy task plus two-stage (pose-then-keypoint) clustering yields better pseudo-labels.

Key results

• D-ULD++ (K=10) reaches forward/backward NME 2.92/3.62 on AFLW and 2.19/2.78 on MAFL, versus 4.98/6.29 and 3.14/3.27 for ZeroShot-type baselines (table reports ZeroShot 3.14/3.27 on MAFL).
• Relative improvement of D-ULD++ over the best published method is above 20% on all four datasets, e.g. 50.91% forward NME gain on AFLW and 59.31% on LS3D.
• Pose-guided proxy task improves AFLW forward/backward NME by 7.8% and 12.4%, and two-stage clustering further lowers errors (Table 2).

Measured effects reported in the paper

Surprising findings

• A simple ZeroShot baseline (clustering of random pixel Stable Diffusion features with nearest-neighbour matching) already outperforms prior published ULD methods on all datasets.

Limitations

• Performance degrades on side-profile and extreme-yaw faces, where ZeroShot confuses left and right keypoints.
• Pose latent space is learned only from landmark heatmaps without pose supervision, so pose clusters are indirect.
• Evaluation relies on regressors trained with a small subset of labeled images for forward and backward NME.

Experimental setup

Datasets: AFLW MAFL LS3D CatHeads

Benchmarks: AFLW MAFL LS3D CatHeads

Metrics: forward NME% backward NME% Silhouette coefficient Calinski-Harabasz index

Baselines: Jakab et al. Zhang et al. Sanchez et al. Mallis et al. Awan et al. Lorenz et al. Shu et al. Mallis (D)

Builds on: Mallis et al. unsupervised landmark pseudo-labelling SILK keypoint learning latent diffusion models Stable Diffusion feature aggregation

Closest papers by meaning

• Object Pose Estimation via the Aggregation of Diffusion Features (CVPR 2024 · similarity 0.85)
• Towards Universal Dataset Distillation via Task-Driven Diffusion (CVPR 2025 · similarity 0.84)
• Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation (CVPR 2025 · similarity 0.84)
• MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models (CVPR 2024 · similarity 0.84)
• Clustering via Self-Supervised Diffusion (ICML 2025 · similarity 0.84)
• SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation (CVPR 2024 · similarity 0.83)