Research map / Label-efficient and robust learning / Recognition and long-tail
What shapes feature representations? Exploring datasets, architectures, and training
Katherine Hermann, Andrew Lampinen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on a target feature rather than leave the model untrained | AlexNet and ResNet-50 · Trifeature and Navon | target-feature decoding accuracy | Improved |
| Train on a target feature, which suppresses unrelated features | AlexNet and ResNet-50 · Trifeature and Navon | non-target feature decoding accuracy | Hurt |
| Prefer color over shape when both redundantly predict the label | AlexNet and ResNet-50 · Trifeature correlated pairs | feature decoding accuracy | Mixed |
| Prefer shape over texture when both redundantly predict the label | AlexNet and ResNet-50 · Trifeature correlated pairs | feature decoding accuracy | Mixed |
| Increase correlation of the non-target feature with the perfectly predictive target | AlexNet and ResNet-50 · Trifeature correlated datasets | correlated non-target feature decoding accuracy | No clear effect |
| Use the easy linear feature as it becomes more predictive | 5-layer MLP · Binary Features | feature reliance | Hurt |
| Train two models on the same task rather than different tasks | Vision models · Trifeature and Navon | representational similarity: 0.408 [0.394, 0.420] versus 0.624 [0.615, 0.632] | Improved |
| Train on texture rather than shape | Vision models · Trifeature | representational similarity: 0.408 [0.394, 0.420] versus 0.624 [0.615, 0.632] | Hurt |
| Train on an easy feature rather than an unpredictive easy feature | Binary-feature models · Binary Features | representational similarity across model runs | Improved |
| Train a multitask model to report both easy and difficult features | Binary-feature model · Binary Features | representational similarity to easy-feature model | No clear effect |
Closest papers by meaning
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.88)
- On the Foundations of Shortcut Learning (ICLR 2024 · similarity 0.86)
- Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.85)
- Evaluating alignment between humans and neural network representations in image-based learning tasks (NeurIPS 2024 · similarity 0.85)
- Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.85)
- Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
