Research map / Learning theory and algorithms / Matrix and tensor methods
Generalized Eigenvalue Problems with Generative Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies generalized eigenvalue problems under Lipschitz generative priors, proving optimal-statistical-rate guarantees for global optima and proposing the Projected Rayleigh Flow Method (PRFM) with linear convergence, validated on MNIST with a VAE.
Problem
High-dimensional generalized eigenvalue problems (GEPs) where the singular or ill-conditioned matrix B-hat prevents sparse-style solvers; the leading generalized eigenvector is assumed to lie in the range of a pre-trained generative model instead of being sparse.
Approach
Replaces the sparsity constraint in sparse GEP / truncated Rayleigh flow with a generative-range constraint, using a projected Rayleigh flow update (gradient step on A-hat minus rho B-hat followed by projection onto the generator range) with a constant step size, and shows statistical-rate and linear-convergence guarantees.
Key results
- PRFM achieves significantly higher cosine similarity than PPower on MNIST reconstruction across measurement counts from 100 to 350 in the spiked setting.
- Rifle20 and Rifle100 give poor reconstructions when the number of measurements is small relative to the ambient dimension n=784.
- PRFM also gives the best reconstructions in the phase-retrieval-style A-hat setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PRFM instead of PPower | MNIST GGEP reconstruction with sample-covariance matrices · MNIST test images | Cosine Similarity | Helped |
| Use PRFM instead of PPower or Rifle | MNIST phase-retrieval GGEP reconstruction · MNIST phase-retrieval model | reconstruction quality | Helped |
| Use Rifle20 or Rifle100 instead of a generative-prior method | MNIST GGEP reconstruction with relatively few measurements · MNIST test images | reconstruction quality | Hurt |
Surprising findings
- PPower performs poorly because it does not handle a non-identity B-hat, which the authors note as the reason for PRFM's advantage.
Limitations
- Projection onto the generator range is assumed exact in theory but only approximated in practice.
- Experiments are proof-of-concept on MNIST with 10 test images.
- Guarantees require an initialization with positive correlation to the signal and step-size conditions on the matrix pair.
Experimental setup
Datasets: MNIST CelebA
Benchmarks: MNIST
Metrics: cosine similarity
Baselines: PPower Rifle20 Rifle100
Builds on: truncated Rayleigh flow projected power method (PPower) Rifle compressed sensing with generative models
Closest papers by meaning
- Scalable First-order Method for Certifying Optimal k-Sparse GLMs (ICML 2025 · similarity 0.84)
- Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization (NeurIPS 2024 · similarity 0.83)
- Preconditioned Langevin Dynamics with Score-based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems (NeurIPS 2025 · similarity 0.83)
- The Generative Leap: Tight Sample Complexity for Efficiently Learning Gaussian Multi-Index Models (NeurIPS 2025 · similarity 0.83)
- Efficient Algorithms for Non-gaussian Single Index Models with Generative Priors (AAAI 2024 · similarity 0.83)
- EigenVI: score-based variational inference with orthogonal function expansions (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.