Research map / Detection and segmentation / Open-set and few-shot detection

Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps

CVPR 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Unsupervised object and part discovery · 21 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adds a weakly supervised spherical 3D prior (viewpoint, relative-distance and orientation losses) on top of frozen DINOv2/DINOv1 features to improve semantic correspondence on SPair-71k, and proposes the KAP metric.

Problem

Self-supervised dense features fail on semantic correspondence under symmetries and repeated parts because they lack 3D understanding and PCK ignores symmetry-induced false matches.

Approach

Map SSL features onto a learned category-conditioned spherical prototype via a spherical mapper trained with coarse viewpoint supervision and geometric constraints (relative distance triplet loss, orientation-preserving loss), and combine sphere coordinates with SSL features at inference; also introduce Keypoint Average Precision (KAP) to penalize spurious matches.

Key results

• On SPair-71k with DINOv2 backbone, the method reaches 63.6 average PCK@0.1 versus 56.2 for DINOv2 alone and 63.3 for DINOv2+SD.
• Adding Stable Diffusion features to the method gives 67.3 average PCK@0.1 on SPair-71k.
• On AwA-pose (unseen categories mostly), the method gets 68.7 PCK@0.1 vs 65.9 for DINOv2, and 58.9 vs 55.0 KAP@0.1.

Measured effects reported in the paper

Surprising findings

• DINOv2 surpasses prior fully-supervised correspondence methods on most SPair-71k categories, though the authors note this may partly reflect evaluation biases.
• The sphere-only representation performs poorly on its own (45.5 PCK) and must be combined with SSL features to be effective.

Limitations

• Performs worse on categories poorly approximated by a sphere, such as humans and bicycles.
• Sphere-only encoding is too low-dimensional to discriminate parts and needs SSL features combined.
• Requires training-time segmentation masks and coarse camera viewpoint supervision.

Experimental setup

Datasets: SPair-71k AwA-pose Freiburg cars

Benchmarks: SPair-71k AwA-pose

Metrics: PCK@0.1 KAP@0.1 throughput (pairs/second)

Baselines: CATS MMNet+FCN SCorrSan DINOv1 ASIC DIFT SD DINOv2 DINOv2 + SD

Builds on: DINOv2 DINO ASIC Stable Diffusion features for correspondence (DIFT/SD+DINO)

Closest papers by meaning

• Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence (NeurIPS 2025 · similarity 0.90)
• Bridging Viewpoint Gaps: Geometric Reasoning Boosts Semantic Correspondence (CVPR 2025 · similarity 0.89)
• SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations (CVPR 2025 · similarity 0.89)
• Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence (CVPR 2024 · similarity 0.88)
• Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose Estimation (ICLR 2025 · similarity 0.87)
• Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence (NeurIPS 2026 · similarity 0.87)