Research map / Architectures and efficiency / CNN design

Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision

NeurIPS 2026 · Architectures and efficiency › CNN design · approach: Biologically inspired CNN explanation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Compares directional confusion asymmetry in humans and deep vision models on the GEN perturbation benchmark, links asymmetry organization to rate-distortion geometry, and uses simulations to show broad-weak and sink-like asymmetries shift RD frontiers in opposite directions.

Problem

Humans and vision models reach similar accuracy but may differ in how their errors are directionally organized; accuracy-based evaluation cannot reveal this, and it is unclear whether asymmetry structure relates to generalization geometry independently of accuracy.

Approach

Treat each system's confusion matrix as a noisy channel, infer a latent distortion matrix via MAP with Blahut-Arimoto, and summarize the rate-distortion frontier (slope, curvature, AUC); decompose asymmetry into breadth (fraction of asymmetric pairs) and strength (mean deviation), contrasting human broad-weak asymmetry with ANN sink-like collapses.

Key results

• ANNs show larger global Frobenius asymmetry than humans (1.22 vs 1.04, p<2.2e-16) but humans have more asymmetric class pairs (85.4 vs 64.2), while ANN per-pair strength is larger (0.141 vs 0.042).
• Distortion-trained models show a strong accuracy-controlled positive asymmetry-AUC association (slope 1.02, p=6.0e-18), while baseline models and humans show no reliable accuracy-controlled slopes.
• In simulations (n=1662 non-collapsed runs), broad-weak generators show positive accuracy-adjusted AUC slopes in all 15 slices while sink slopes are near zero or negative; strict recovery passes 30.6% of runs removed.

Surprising findings

• Groups significantly more asymmetric than humans show no significant accuracy difference from humans, while groups matching humans on asymmetry are more accurate, a double dissociation.
• Accuracy-controlled Distortion-trained asymmetry-AUC association is positive, reversing the sign of the pooled negative correlation (-0.73).

Limitations

• Analyses restricted to K=16 class categorization from GEN benchmark; generalization to other label granularities untested.
• Human sample (n=81 blocks) much smaller than ANN sample (n=1569 blocks).
• MAP inference of rho validated against ground truth only in simulation; prior sensitivity in empirical setting not assessed.

Experimental setup

Datasets: GEN model-vs-human benchmark (~83k human psychophysics trials, 12 perturbation families, K=16 ImageNet-derived classes) ModelZoo confusion matrices

Benchmarks: GEN model-vs-human benchmark ModelZoo (15 modern architectures)

Metrics: Frobenius asymmetry index breadth (fraction of asymmetric pairs) strength (conditional mean deviation) RD slope RD curvature RD AUC recovery fraction

Baselines: GoogLeNet ResNet-152 VGG-19 Distortion-trained ResNet-50 Specialised models All-noise models

Builds on: Blahut-Arimoto algorithm GEN model-vs-human benchmark rate-distortion fitting of confusion matrices

Closest papers by meaning

• Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.87)
• Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.87)
• Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.87)
• Contour Integration Underlies Human-Like Vision (ICML 2025 · similarity 0.86)
• Discriminating image representations with principal distortions (ICLR 2025 · similarity 0.86)
• Vision Language Models are Biased (ICLR 2026 · similarity 0.86)