Research map / Architectures and efficiency / RNNs and normalization
Understanding Visual Feature Reliance through the Lens of Complexity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a V-information-based complexity metric for ~10,000 dictionary-learned features in an ImageNet ResNet50 and analyzes where, when, and how important those features are across layers and training.
Problem
Prior work shows neural networks favor simple features (simplicity bias, shortcut learning), but there is no quantitative framework for measuring the complexity of the many features learned by large natural-image vision models.
Approach
Define feature complexity as one minus the average V-information (with linear probe predictive family) of a feature across all layers of the network, and apply it to over-complete NMF/CRAFT-style concept features from the penultimate layer to study what, where, when, and how important complex features are.
Key results
- ResNet50 trained from scratch reaches 78.9% ImageNet validation accuracy; complexity spans a spectrum from color/low-frequency features (simple) to whiskers, insect legs, ears (complex).
- Correlation near 0.5 between feature complexity and Time to Decode score: complex features emerge later in training.
- Simple features are teleported to late layers via residual branches (CKA), while complex features are built gradually through the main branch; complex features are generally less important in the final epoch, while important features shift to earlier layers during training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dictionary-based feature extraction with over-complete dictionary (10,000 features vs 2048 neurons) on penultimate layer | ResNet50 penultimate layer · ImageNet | qualitative feature complexity spectrum | No clear effect |
| Residual connections carry simple features forward to the final layer (teleportation) while complex features are built via main branch | ResNet50 residual blocks · ImageNet feature flow analysis (CKA) | CKA between residual activations and feature values | Helped |
| Training progress: complex features are learned later in training than simple ones | ResNet50 feature dynamics · ImageNet training epochs | correlation between complexity and time to decode: correlation nearing 0.5 | Helped |
| Training progress: simplicity bias emerges (complex features get lower importance) by end of training, not at epoch 1 | ResNet50 features · ImageNet feature importance | importance vs complexity | Helped |
| Complex features are less redundant (lower redundancy score) and less robust to Gaussian input noise | ResNet50 features · ImageNet validation | redundancy and sensitivity (variance under noise) | No clear effect |
Surprising findings
- Important features become decodable at earlier layers during training (a 'sedimentation' effect), so the model reduces complexity of its most important features after an initial rise.
- Most complex features are individually low-importance yet removing them progressively degrades performance ('support features').
Limitations
- Analysis restricted to a single ResNet50 trained on ImageNet, limiting generality across architectures.
- Assumes each layer provides the optimal representation for the linear probe; violated early in training, possibly overestimating complexity.
- Complexity measure based on linear Gaussian V-information family, with features defined via NMF dictionary learning on penultimate layer only.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet validation set
Metrics: V-information complexity score Time to Decode score CKA gradient-input importance top-1 accuracy
Builds on: V-information CRAFT Centered Kernel Alignment gradient-input attribution
Closest papers by meaning
- Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory (ICLR 2024 · similarity 0.87)
- Complexity Matters: Feature Learning in the Presence of Spurious Correlations (ICML 2024 · similarity 0.86)
- Don't trust your eyes: on the (un)reliability of feature visualizations (ICML 2024 · similarity 0.86)
- On the Foundations of Shortcut Learning (ICLR 2024 · similarity 0.86)
- The Impact of Geometric Complexity on Neural Collapse in Transfer Learning (NeurIPS 2024 · similarity 0.85)
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.