Research map / Learning theory and algorithms / Matrix and tensor methods

Learning to Discover Iterative Spectral Algorithms

NeurIPS 2026 · Learning theory and algorithms › Matrix and tensor methods · approach: Geometry of low-rank and SPD structure · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoSpec, a self-supervised neural engine that maps coarse spectral probes (eigenvalue estimates and residuals) to recurrence coefficients for matrix polynomials, trained on small synthetic diagonal spectra and applied to large SPD matrices for linear solves, eigenproblems, and inverse square roots.

Problem

Classical polynomial and iterative spectral methods for large sparse SPD problems need spectral information to be tuned, and discrete program-search discovery frameworks do not transfer to continuous NLA algorithm design.

Approach

Train a neural network that maps a cheap spectral probe to coefficients of an executable three-term-style matrix-free recurrence, optimizing task-level convergence-ratio objectives end-to-end on small synthetic diagonal spectra, then transfer to real SuiteSparse matrices without retraining.

Key results

• Learned polynomial preconditioners for CG achieve order-of-magnitude reductions in iterations versus Neumann preconditioning across SuiteSparse test cases.
• For eigenproblems, AutoSpec preconditioning of eigs consistently outperforms the power-preconditioner baseline across (k, l) settings and matrices.
• For inverse square root whitening of the E. coli Gram matrix, the learned degree-21 polynomial achieves order-of-magnitude lower operator-norm error than Neumann series.

Surprising findings

• Learned polynomials show near-minimax equioscillation behavior analogous to Chebyshev polynomials, despite no explicit Chebyshev supervision.
• Training on small synthetic diagonal matrices (n up to about 1e3) transfers to real matrices up to about 1.6e6 in size.

Limitations

• Evaluation restricted to symmetric positive-definite problems; indefinite and unsymmetric cases left to future work.
• Requires a synthetic training spectrum set covering the spectral features of test matrices; performance degrades if training eigengaps are larger than at test time.
• Efficiency measured by outer iteration counts rather than wall-clock runtime.

Experimental setup

Datasets: SuiteSparse Matrix Collection synthetic eigenspectra E. coli K-12 MG1655 DNA k-mer Gram matrix

Benchmarks: SuiteSparse matrices (linear systems, eigenproblems) E. coli DNA whitening Gram matrix

Metrics: CG iteration count to 1e-10 relative residual condition number of preconditioned operator eigs outer iterations / eigenvalue convergence operator-norm whitening residual minimax optimality gap

Baselines: Neumann polynomial preconditioner Richardson iteration power/subspace iteration (eigs with power preconditioner) Chebyshev polynomial preconditioner

Builds on: Chebyshev polynomial preconditioning Neumann series Krylov-Schur eigensolver Clenshaw recurrence

Closest papers by meaning

• Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.87)
• Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.86)
• STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem (NeurIPS 2025 · similarity 0.86)
• Revisiting Orbital Minimization Method for Neural Operator Decomposition (NeurIPS 2025 · similarity 0.86)
• Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.85)
• Neural Spectral Methods: Self-supervised learning in the spectral domain (ICLR 2024 · similarity 0.85)