Research map / Vision-language and multimodal / Multimodal LLMs
SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
Olaf Dünkel, Basavaraj Sunagad, Haoran Wang, David Hoffmann, Christian Theobalt, Adam Kortylewski
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| require object-relative identity (SOC) instead of concept matching only (CC) | vision foundation models (all evaluated) · SOCO | PCK@0.1: DINOv2 78.9 -> 60.4 | Hurt |
| move from within-category SOC to cross-category SOC | DINOv2 · SOCO | PCK@0.1: 60.4 -> 54.9 | Hurt |
| dense self-supervised objective (DINOv2) versus global alignment (CLIP) | vision encoders · SOCO SOC | PCK@0.1: 16.2 -> 60.4 | Improved |
| scale DINOv2 to DINOv3 | DINO · SOCO SOC | PCK@0.1: SOC 60.4 -> 55.5; SOC-geo 71.9 -> 76.0 | Mixed |
| use reconstruction objective (MAE) instead of dense SSL | vision encoders · SOCO SOC | PCK@0.1: DINOv2 60.4 -> MAE 9.4 | Hurt |
| supply keypoint text description only instead of visual marker | Qwen3-VL-8B · SOCO LVLM | CircularEval accuracy: 34.2 -> 54.0 | Improved |
| add description to visual query | Qwen3-VL-8B · SOCO LVLM | CircularEval accuracy: 34.2 -> 30.8 | Hurt |
| add description to visual query | GPT4o · SOCO LVLM | CircularEval accuracy: 30.2 -> 30.9 | No clear effect |
| use DINOv2 instead of best LVLM in 4-choice protocol | DINOv2 · SOCO LVLM | CircularEval accuracy: 54.0 (best LVLM) -> 81.0 | Improved |
| scale Qwen3-VL from 4B to 8B and compare with Qwen2.5-VL-7B | Qwen · SOCO LVLM (Vis.) | CircularEval accuracy: Qwen2.5-VL-7B 19.4 -> Qwen3-VL-8B 34.2 | Improved |
| train a linear probe on features instead of nearest-neighbor matching (ablation) | vision encoders · SOCO SOC | PCK@0.1: DINOv2 60.4 -> 69.8 | Improved |
| use window soft-argmax evaluation (ablation) | vision encoders · SOCO SOC | PCK@0.1: DINOv2 60.4 -> 62.7 | Improved |
Closest papers by meaning
- Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.88)
- Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence (NeurIPS 2025 · similarity 0.88)
- Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence (CVPR 2024 · similarity 0.88)
- Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.88)
- CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Models (CVPR 2025 · similarity 0.88)
- SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
