Research map / Architectures and efficiency / RNNs and normalization
Intrinsic dimension of data representations in deep neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CNNs on image classification tasks | CNN representations across hidden layers · ImageNet and custom object images | intrinsic dimension across layers | Mixed |
| Reduce intrinsic dimension in the last hidden layer | Trained image-classification networks · ImageNet | test top-5 score: r = 0.94 | Helped |
| Replace TwoNN with PCA-based dimensionality using 90% variance (ablation) | VGG-16 representations · VGG-16, last hidden layer | dimensionality estimate: ≈ 18 -> ≈ 200 | Mixed |
| Use a randomly initialized network instead of a trained network (ablation) | VGG-16 · Object manifolds across layers | intrinsic dimension profile | Hurt |
| Train on randomly shuffled labels instead of correct MNIST labels (ablation) | Small convolutional network · MNIST† | intrinsic dimension in final layers | Hurt |
| Add a shared per-image luminance perturbation (ablation) | MNIST · MNIST★ input representation | intrinsic dimension: ≈ 13 -> ≈ 3 | Hurt |
| Train on MNIST★ with the luminance perturbation (ablation) | Small convolutional network · MNIST★ | test accuracy: ≈ 98 % | No clear effect |
| Vary random weight initialization (ablation) | VGG-16 adapted for CIFAR-10 · CIFAR-10 | correlation between last-layer ID and accuracy: r=-0.003 | No clear effect |
| Use a synthetic Gaussian dataset with the same second-order moments (ablation) | VGG-16 last-hidden-layer representations · Synthetic Gaussian comparison | intrinsic dimension | Hurt |
| Monitor training with a smaller learning rate (ablation) | VGG-16 adapted for CIFAR-10 · CIFAR-10 training dynamics | last-hidden-layer intrinsic dimension dynamics | Mixed |
Closest papers by meaning
- The Intrinsic Dimension of Images and Its Impact on Learning (ICLR 2021 · similarity 0.87)
- The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images (ICLR 2024 · similarity 0.87)
- Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations (ICLR 2025 · similarity 0.86)
- When Representations Align: Universality in Representation Learning Dynamics (ICML 2024 · similarity 0.86)
- How Data Augmentation Shapes Neural Representations (NeurIPS 2026 · similarity 0.85)
- Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.