Research map / Learning theory and algorithms / Kernels and regression theory
On the Limitations of Fractal Dimension as a Measure of Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirically re-evaluates persistent homology (PH) dimension-based generalization measures, using hyperparameter-grid partial correlations, conditional independence tests, adversarial initialization, and CNN width sweeps to show their limitations.
Problem
Whether PH-based fractal dimensions of optimization trajectories predict and causally relate to the generalization gap of overparameterized neural networks, as theory (fractal/topological generalization bounds) suggests.
Approach
Stress-tests PH dimension generalization measures with partial correlations controlling for learning rate/batch size, conditional mutual information independence tests, adversarial initialization, and model-wise double descent, revealing confounding and failure modes not examined by prior work.
Key results
- Partial correlations show learning rate significantly influences Euclidean PH dimension vs generalization correlation for most batch sizes, while loss-based dimension is less affected.
- CMI tests find Euclidean and loss-based PH dimension conditionally independent of generalization gap given learning rate on MNIST for most batch sizes, but conditionally dependent on CHD.
- Under adversarial initialization, PH dimensions fail for CNN CIFAR-10 and FCN-5 MNIST (lower PH dimension despite higher generalization gap); Euclidean PH dimension shows double descent with test accuracy while generalization gap is monotonic.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use loss-based rather than Euclidean PH dimension as the generalization-gap measure | FCN-5 and FCN-7 · CHD and MNIST grid experiments | Spearman's ρ with generalization gap: 0.71 -> 0.78; 0.45 -> 0.67; 0.67 -> 0.77; 0.78 -> 0.88 | Helped |
| Use loss-based rather than Euclidean PH dimension as the generalization-gap measure | AlexNet · CIFAR-10 grid experiments | Spearman's ρ with generalization gap: 0.85 -> -0.31 | Hurt |
| Use PH dimension as a generalization-gap indicator across learning-rate and batch-size settings | Euclidean and loss-based PH dimensions · MNIST | Conditional independence test p-value | Hurt |
| Train from adversarial rather than random initialization | CNN and FCN-5 · CIFAR-10 and MNIST | PH dimension versus generalization gap | Hurt |
| Train from adversarial rather than random initialization | AlexNet · CIFAR-10 | PH dimension versus generalization gap | Helped |
| Vary model width through the double-descent region | Standard CNN · CIFAR-100 | Correlation with generalization gap | Hurt |
| Vary model width through the double-descent region | Euclidean PH dimension · CIFAR-100 | Double-descent pattern | Helped |
Surprising findings
- Loss-based PH dimension on AlexNet CIFAR-10 is negatively correlated with generalization gap, with high learning rate models getting high dimension despite small gap.
- Models from adversarial initialization get lower PH dimension despite higher generalization gap on several settings.
Limitations
- Restricted to vanilla SGD, constant learning rate, no batch norm or explicit regularization to match theory assumptions.
- Limited and unconventional hyperparameter grid; few seeds for some experiments (e.g., 3 seeds for double descent).
- No identified pattern explaining when and why PH dimension fails to correlate with generalization.
Experimental setup
Datasets: California housing MNIST CIFAR-10 CIFAR-100
Benchmarks: California housing MNIST CIFAR-10 CIFAR-100
Metrics: Spearman rho mean granulated Kendall Psi Kendall tau partial correlation conditional mutual information p-value generalization gap test accuracy
Baselines: Birdal et al. PH dimension measure Euclidean PH dimension loss-based PH dimension parameter norm
Builds on: fractal dimension generalization bounds (Simsekli et al.) PH dimension generalization bounds (Birdal et al.) data-dependent pseudometric PH bounds (Dupuis et al.) adversarial initialization (Zhang et al.)
Closest papers by meaning
- Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks (NeurIPS 2021 · similarity 0.95)
- Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms (NeurIPS 2024 · similarity 0.92)
- Generalization at the Edge of Stability (NeurIPS 2026 · similarity 0.89)
- Compositional PAC-Bayes: Generalization of GNNs with persistence and beyond (NeurIPS 2024 · similarity 0.86)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.86)
- Towards Scalable Topological Regularizers (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.