Research map / Learning theory and algorithms / Matrix and tensor methods
Adaptive Randomized Dimension Reduction on Massive Data
Gregory Darnell, Stoyan Georgiev, Sayan Mukherjee, Barbara E Engelhardt
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add more Krylov iterations t to ARSVD (ablation) | ARSVD · Singular value estimation (2000x5000, rank 50) | percent relative error: 26.1 -> 0.3 (t=1 to t=5) | Improved |
| use ARSVD instead of Blocked Lanczos at matched reconstruction error | ARSVD · Run time ratio, rank 50 | run time ratio: 2.5 -> 1.84 (n+p 6,000 to 12,000) | No clear effect |
| use ARSVD instead of exact eigendecomposition for WTCCC genome-wide covariance | pylmm · WTCCC | decomposition time: 88 mins 23.9 sec -> 82.2 secs | Improved |
| use ARSVD for 22 leave-one-chromosome-out decompositions | pylmm · WTCCC | decomposition time: 4 hrs 24 mins -> 5 mins 4.8 secs | Improved |
| use ARSVD low-rank covariance in LMM instead of full-rank LMM, p>>n or n=p | LMM association · simulated null with latent population structure | false positive rate | No clear effect |
| use ARSVD low-rank covariance in LMM in over-parametrized case n=5,000, p=1,000 | LMM association · simulated null with latent population structure | false positive rate | Hurt |
| use ARSVD with rank close to full rank in LMM, n=1,000 p=1,000 | LMM association · simulated null with latent population structure | p-value uniformity | Improved |
| use adaptive rank estimation with kappa=1 instead of kappa=2 (ablation) | ARSVD · rank estimation, 50 random datasets | rank estimation accuracy | Hurt |
| use ARSVD LMM (pylmm) instead of GEMMA | LMM association · WTCCC Crohn's disease | enrichment of significant p-values | Improved |
Closest papers by meaning
- Capturing the denoising effect of PCA via compression ratio (NeurIPS 2024 · similarity 0.85)
- Heterogeneous Sufficient Dimension Reduction and Subspace Clustering (ICML 2025 · similarity 0.84)
- No Double Descent in Principal Component Regression: A High-Dimensional Analysis (ICML 2024 · similarity 0.84)
- Efficient Sparse PCA via Block-Diagonalization (ICLR 2025 · similarity 0.83)
- Dimension Reduction with Locally Adjusted Graphs (AAAI 2025 · similarity 0.83)
- Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
