Research map / Learning theory and algorithms / Matrix and tensor methods
Spectral Estimation with Free Decompression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces "free decompression," which uses free probability and a PDE on the Stieltjes transform to extrapolate the spectral density of a huge impalpable matrix from small random submatrices, with the freealg Python package.
Problem
Estimating the eigenspectrum (spectral density) of extremely large matrices that cannot be formed or accessed via matrix-vector products, where only small submatrices are available.
Approach
Treats the Stieltjes transform of a random submatrix as evolving under a first-order quasilinear PDE in the log size ratio (derived from free compression/Nica-Speicher), solved by characteristics with analytic continuation via a gluing function, to extrapolate the spectrum to a larger matrix.
Key results
- On the NTK example, free decompression from a 2048x2048 submatrix matches the 50000x50000 spectrum with TV distance 2.9% and JS divergence 5.5% at 50K, versus 67331 s direct computation (about 26 s for FD after the initial 10.2 s).
- For Marchenko-Pastur, decompressing from n=1000 to n=32000 yields average TV distance 0.2% and log-determinant relative error 1.78% across ten matrices.
- Free decompression recovers Wigner, Kesten-McKay, Wachter and Meixner laws from small submatrices by analytic formulas and PDE evolution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use free decompression instead of direct computation to estimate larger NTK spectral densities | ResNet50 CIFAR-10 NTK · NTK, from 2^12 through 50K | process time (sec): 50.9 vs 10.2 + 54.2; 67331.1 vs 10.2 + 16.2 | Helped |
| Estimate larger NTK spectra by free decompression from the 2^11 submatrix | ResNet50 CIFAR-10 NTK · NTK, sizes 2^12 through 50K | TV: 1.2%; 1.9%; 2.4%; 2.6%; 2.9% | Mixed |
| Decompress Marchenko–Pastur estimates from size 1000 to size 32000 | Free decompression · Marchenko–Pastur synthetic benchmark | TV: 0.2% | Helped |
| Decompress Marchenko–Pastur estimates from size 1000 to size 32000 | Free decompression · Marchenko–Pastur synthetic benchmark | Jensen–Shannon divergence: 1.867% | Helped |
| Decompress Marchenko–Pastur estimates from size 1000 to size 32000 | Free decompression · Marchenko–Pastur synthetic benchmark | log-determinant: -13514.88; 1.503; -13538.94; 1.78%; 0.01% | Mixed |
Surprising findings
- Free decompression reaches large-size spectra with small error even when naive empirical spectral distributions cannot be extended, as the Stieltjes transform itself cannot be evolved directly.
Limitations
- Accuracy depends strongly on the quality of the spectral density estimate and gluing function, becoming delicate on real data.
- Ill-conditioned matrices remain hard, since estimating the Stieltjes transform on log-scale is not yet general.
- Requires asymptotic freeness and large matrix assumptions.
Experimental setup
Datasets: SNAP Facebook Page-Page network CIFAR-10 synthetic Wishart/Gaussian random matrices
Benchmarks: Marchenko-Pastur law Wigner semicircle Kesten-McKay Wachter Meixner SNAP Facebook Page-Page Laplacian spectrum empirical ResNet-50 CIFAR-10 NTK spectrum
Metrics: total variation distance Jensen-Shannon divergence relative error in first and second moments log-determinant error runtime
Baselines: direct eigendecomposition of submatrices Nystrom method FLODANCE
Builds on: free compression Nica-Speicher theorem Marchenko-Pastur law Hutchinson trace estimation stochastic Lanczos quadrature
Closest papers by meaning
- Spectral Estimation with Deformed Decompression (NeurIPS 2026 · similarity 0.91)
- Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.90)
- Determinant Estimation under Memory Constraints and Neural Scaling Laws (ICML 2025 · similarity 0.86)
- Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy (NeurIPS 2025 · similarity 0.86)
- Nonlinear Laplacians: Tunable principal component analysis under directional prior information (NeurIPS 2025 · similarity 0.86)
- Entrywise error bounds for low-rank approximations of kernel matrices (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.