Research map / Detection and segmentation / Open-set and few-shot detection
Perceptual Inductive Bias Is What You Need Before Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a pre-pretraining stage using figure-ground shape prototypes and intrinsic images (reflectance/shading) as perceptual inductive biases before standard contrastive learning, yielding 2x faster convergence on ResNet18 and gains on classification, segmentation, depth, and shape bias.
Problem
Contrastive self-supervised learning bypasses Marr's multi-stage visual processing, leading to slow convergence and texture bias, and a trade-off between image-level semantic and pixel-level perceptual tasks.
Approach
Inject early-vision perceptual constructs (figure-ground shape silhouettes clustered into shape prototypes, and Retinex intrinsic reflectance/shading images as extra views) into contrastive learning, and use a shape-prototype warm-up stage before standard contrastive training (S-PCL+X).
Key results
- S-PCL+PCL reaches accuracy comparable to PCL while requiring less than half the training epochs on ImageNet-100 and STL-10 (claimed 2x faster convergence).
- MidVCL achieves 32.1 mIoU on ADE20K and 68.3 on Cityscapes at 400 epochs, above SimCLR (30.4 / 63.4).
- S-PCL is the only model with shape bias score above 0.5 on the model-vs-human benchmark; S-PCL+MoCoV2 reaches 78.0 Top-1 on ImageNet-100 at 400 epochs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shape prototypes during contrastive pretraining | S-PCL · ImageNet-100, 100 epochs | Top-1 accuracy (%): 65.8 -> 70.2 | Helped |
| Use shape prototypes throughout 400-epoch training | S-PCL · ImageNet-100, 400 epochs | Top-1 accuracy (%): 77.2 -> 71.8 | Hurt |
| Use reflectance images as an additional contrastive view | ReflCL · ImageNet-100, 400 epochs | Top-1 accuracy (%): 77.2 -> 77.2 | No clear effect |
| Use shading images as an additional contrastive view | ShadCL · ImageNet-100, 400 epochs | Top-1 accuracy (%): 77.2 -> 71.8 | Hurt |
| Combine shape prototypes, reflectance, and shading inductive biases | MidVCL · ADE20K, 400 epochs | mIoU (%): 30.9 -> 32.1 | Helped |
| Use shape prototypes as an inductive bias | S-PCL · NYU Depth V2, 400 epochs | REL: 0.2543 -> 0.2428 | Helped |
| Use reflectance images as an additional contrastive view | ReflCL · NYU Depth V2, 400 epochs | REL: 0.2543 -> 0.2533 | No clear effect |
| Use shading images as an additional contrastive view | ShadCL · NYU Depth V2, 400 epochs | REL: 0.2543 -> 0.2450 | Helped |
| Pretrain with S-PCL for 100 epochs, then train with MoCoV2 | S-PCL+MoCoV2 · ImageNet-100, 400 epochs | Top-1 accuracy (%): 77.0 -> 78.0 | Helped |
| Pretrain with S-PCL for 100 epochs, then train with PCL | S-PCL+PCL · ADE20K, 400 epochs | mIoU (%): 30.5 -> 31.9 | Helped |
| Pretrain with perceptual shape bias before semantic contrastive learning | S-PCL+PCL · ImageNet-100 and STL-10 | training epochs to comparable accuracy: less than half the number of training epochs | Helped |
| Use shape prototypes during contrastive pretraining | S-PCL · 17 OOD datasets in the model-vs-human benchmark | average classification accuracy | Helped |
Surprising findings
- Shape-only clustering's alignment with semantic categories (AMI) decreases in late training, so the shape bias must be disabled after warm-up to avoid hurting semantic representations.
- Reflectance intrinsic images help classification and segmentation but not depth estimation, while shading images help depth but not classification or segmentation.
Limitations
- Experiments limited to a small model (ResNet18) and limited datasets due to compute constraints.
- Only figure-ground shape, reflectance, and shading constructs explored; other mid-level perceptual constructs not studied.
- Shape-only clustering diverges from semantic categories in late training, requiring a hybrid warm-up schedule.
Experimental setup
Datasets: ImageNet-100 STL-10 ImageNet-1k ADE20K Cityscapes NYU Depth V2 model-vs-human benchmark (17 OOD datasets)
Benchmarks: ImageNet-100 linear classification ImageNet-1k transfer ADE20K Cityscapes NYU Depth V2 model-vs-human shape bias model-vs-human OOD
Metrics: Top-1 accuracy Top-5 accuracy mIoU REL RME shape bias fraction OOD accuracy AMI
Baselines: SimCLR MoCoV2 PCL SwAV BYOL DINO
Builds on: Prototypical Contrastive Learning InfoNCE MoCo v2 Retinex
Closest papers by meaning
- Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models (ICLR 2025 · similarity 0.87)
- Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation (AAAI 2026 · similarity 0.87)
- What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? (ICLR 2025 · similarity 0.87)
- Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception (CVPR 2024 · similarity 0.87)
- Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.86)
- FFF: Fixing Flawed Foundations in Contrastive Pre-Training Results in Very Strong Vision-Language Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.