AI Paper Atlas

Research map / 3D vision / Depth and stereo

CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion

Philippe Weinzaepfel, Vincent Leroy, Thomas Lucas, Romain Brégier, Yohann Cabon, Vaibhav ARORA, Leonid Antsfeld, Boris Chidlovskii, Gabriela Csurka, Jerome Revaud

NeurIPS 2022 · 3D vision › Depth and stereo · approach: Foundation models for monocular geometry · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
cross-view completion pre-training instead of single-view MAE on the same Habitat data (ablation)ViT-Base/16 · NYUv2 depthdelta_1: 79.0 -> 85.6Improved
cross-view completion instead of MAE on same Habitat data (ablation)ViT-Base/16 · Taskonomy average L1L1 x1000: 35.65 -> 33.00Improved
cross-view completion instead of MAE on same Habitat data (ablation)ViT-Base/16 · ADE20k segmentationmIoU: 40.3 -> 40.6No clear effect
normalize reconstruction targets per patch (ablation)CroCo CrossBlock · NYUv2, ADE, Taskonomydelta_1 / mIoU / L1: 83.4 -> 85.6 (NYUv2); 39.0 -> 40.6 (ADE); 34.74 -> 33.00 (Taskonomy)Improved
use CatBlock instead of CrossBlock decoder (ablation)CroCo · NYUv2 / ADE / Taskonomydelta_1 / mIoU / L1: 85.6 -> 86.2 (NYUv2); 33.00 -> 33.35 (Taskonomy)Mixed
use two real viewpoints instead of geometric transformations of one image to build pairs (ablation)CroCo · NYUv2 depthdelta_1: 66.1 -> 86.8Improved
use 90% masking ratio (ablation)CroCo CrossBlock · ADE, NYUv2, Taskonomydownstream performanceImproved
keep pre-trained decoder when finetuning on monocular tasks (ablation)CroCo · NYUv2 depthdelta_1: 86.1 -> 88.1Improved
pre-train on Habitat indoor scenes instead of ImageNet-1KMAE · ImageNet-1K linear probingtop-1 accuracy: 68.0 -> 32.5Hurt
initialize with CroCo encoder and decoder instead of MAE (Habitat) encoderoptical flow regression · MPI-Sintel cleanAEPE: 4.63 -> 3.00Improved
CroCo pre-training instead of random initoptical flow regression · MPI-Sintel cleanAEPE: 18.81 -> 3.00Improved
CroCo init instead of MAE (Habitat) encoder with scratch decoderrelative pose regression · 7-scenesmedian position error: 24.8cm -> 5.0cmImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy