Research map / Detection and segmentation / Open-set and few-shot detection
Contour Integration Underlies Human-Like Vision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a fragmented-object contour integration benchmark (phosphene vs segment stimuli) and compares 50 humans with over 1,000 vision models, finding that integration bias tracks training data scale and that training on fragmented ImageNet raises shape bias.
Problem
Models generalize worse than humans to new input distributions, and existing benchmarks do not pinpoint where and why models fail at contour integration, a hallmark of human vision.
Approach
A controlled psychophysics-style benchmark with graded object fragmentation (9 levels) and two element types (directionless phosphenes vs directional segments) is used to define an integration bias, which is then correlated with model training data scale and used as a training target to induce shape bias.
Key results
- Human accuracy averaged 0.59 across conditions on the fragmented task, while most of over 1,000 models barely exceeded chance zero-shot; GPT-4o was the best zero-shot model near human level.
- Training dataset size correlated with model performance at r=0.814 and with integration bias at r=0.594 across 1,038 decoder-fit models.
- ResNet-18 trained on ImageNet plus phosphene and segment fragments showed higher shape bias than a Stylized ImageNet-trained ResNet-50, while losing some full-colour RGB accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase training dataset size | Decoder-fit vision models · Fragmented object recognition task | accuracy: r=0.814, p<0.001 | Helped |
| Increase compute per sample | Vision models · Fragmented object recognition task | accuracy: r=0.334, p<0.001 | Helped |
| Use decoder fitting instead of zero-shot category mapping | Pre-trained vision models · Fragmented object recognition task | accuracy | Helped |
| Use directional segments instead of directionless phosphenes | Humans and vision models · Fragmented object recognition task | accuracy: Cohen's d=-1.96; Cohen's d=-0.84; Cohen's d=-1.55 | Helped |
| Increase integration bias | Vision models · Fragmented object recognition task | accuracy | Helped |
| Increase integration bias | Vision models · ImageNet-C-top1 | accuracy: r=0.671, p<0.001 | Helped |
| Increase integration bias | Vision models · ImageNet top1 | accuracy: r=0.67, p<0.01 | Helped |
| Train on ImageNet plus fragmented segment or phosphene stimuli | ResNet-18 · Fragmented object recognition task | accuracy | Helped |
| Train on fragmented segment or phosphene stimuli | ResNet-18 · RGB object recognition | accuracy | Hurt |
| Train on contour-fragmented stimuli | ResNet models · Shape-bias cue-conflict task | cue conflict accuracy | Helped |
| Train solely on contour-extracted images (ablation) | ResNet models · Integration and shape bias | bias | No clear effect |
| Compare CNNs with ViTs | ImageNet-1k-trained models · Fragmented object recognition task | accuracy: CNNs: 32.49%; ViTs: 31.24%; p=0.1108 | No clear effect |
Surprising findings
- Models exhibited a human-like preference for directional segments over directionless phosphenes even though their overall accuracy was far below humans.
- Training on fragmented ImageNet improved fragmented-object performance but reduced full-colour RGB recognition, unlike previous studies.
Limitations
- Observational findings only; no causal mechanism established.
- Vision-language models were not evaluated at the same scale as vision-only models.
- Stimuli were not public, and possible leakage through pre-trained model training sets could not be fully excluded.
Experimental setup
Datasets: BOSS ImageNet-1k ImageNet-21k EcoSet ImageNet-C Stylized ImageNet cue conflict
Benchmarks: fragmented object recognition (12-AFC, 9 fragmentation levels) ImageNet-C top-1 ImageNet top-1 cue conflict shape bias
Metrics: accuracy integration bias Cohen's d Pearson correlation
Baselines: GPT-4o Stylized ImageNet-trained ResNet-50 ResNet-18 ImageNet-1k baseline
Builds on: Brain-Score ImageNet-C Stylized ImageNet
Closest papers by meaning
- Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.89)
- Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models (NeurIPS 2025 · similarity 0.87)
- Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models (NeurIPS 2024 · similarity 0.86)
- Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision (NeurIPS 2026 · similarity 0.86)
- Intriguing Properties of Generative Classifiers (ICLR 2024 · similarity 0.85)
- The 3D-PC: a benchmark for visual perspective taking in humans and machines (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.