Research map / Learning theory and algorithms / Matrix and tensor methods

Why Learning Rediscovers the Closed-Form Diagonal Regularizer

NeurIPS 2026 · Learning theory and algorithms › Matrix and tensor methods · approach: Low-rank approximation theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Argues that under approximately isotropic truncation noise (Berry's conjecture plus Weyl's law), the Bayes-optimal diagonal Tikhonov penalty is a closed-form power law Γ_k ∝ λ_k^|s| set by the prior, and shows on simulated acoustic rooms that learned diagonal regularizers do not beat it, while a non-diagonal learned method (LIR) does.

Problem

Choosing the shape of a diagonal Tikhonov regularizer for modal inverse problems (e.g., recovering room acoustic modal amplitudes from sparse microphones) under truncation noise from discarded eigenmodes.

Approach

Combines a Bayesian derivation (prior-determined power-law shape under isotropic noise) with Berry's random-wave decorrelation and Weyl's counting law to argue the noise is approximately isotropic, so the closed-form power law is near-optimal across rooms and learned diagonal regularizers saturate at it.

Key results

• Closed-form Gamma_k ∝ λ_k^|s| (|s|=1.13) has median relative cost ≤5.82% vs per-room oracle across observation windows; 68.4% of in-scope rooms under 1.1 pp absolute cost.
• M3 learned diagonal regularizer matches ridge at |s| within 0.31 pp at every T (e.g., P=0.1339 vs 0.1308 at T=1000).
• Learned Iterative Ridge (non-diagonal) improves over the diagonal oracle by 9.4, 13.5, and 1.9 pp at T=1, 100, 1000 (L=10).

Surprising findings

• Five seeds of the M3 learned regularizer learn qualitatively different spectra yet reach identical reconstruction error.
• Per-room cost correlates negatively with anisotropy (Spearman rho=-0.30), contrary to the intuition that anisotropy hurts.

Limitations

• Synthetic data dominates; real-data pilot in one compact room recovers slope 0.83 vs 1.13 population value.
• Rooms are 2D random convex polygons; 3D extension left as future work.
• Isotropy is only approximate (median ||E||_op 0.58, mean 0.88), so the closed form is not exactly Bayes-optimal; the claim is restricted to the diagonal family.

Experimental setup

Datasets: FEM-simulated acoustic rooms (197 random convex polygons) synthetic heat diffusion data

Benchmarks: simulated acoustic rooms (197 polygons, 187 in-scope) heat diffusion synthetic data

Metrics: normalized modal MSE P relative cost delta vs per-room oracle KS statistic vs chi-squared(1)

Baselines: per-room oracle diagonal Tikhonov ridge at |s| Wiener/LMMSE generalized Tikhonov TSVD CGLS Landweber Pinsker estimator empirical Bayes

Builds on: Tikhonov regularization Berry's random-wave conjecture Weyl's law algorithm unrolling James-Stein shrinkage

Closest papers by meaning

• Learning Cocoercive Conservative Denoisers via Helmholtz Decomposition for Poisson Imaging Inverse Problems (NeurIPS 2025 · similarity 0.84)
• SBNO : Schrödinger Bridge Neural Operator for the Forward and Inverse Problems under Missing Information (NeurIPS 2026 · similarity 0.84)
• Guided Diffusion Sampling on Function Spaces with Applications to PDEs (NeurIPS 2025 · similarity 0.84)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.84)
• Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.84)
• Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models (ICML 2024 · similarity 0.83)