Research map / Learning theory and algorithms / Matrix and tensor methods

Bivariate Matrix-valued Linear Regression (BMLR): Finite-sample performance under Identifiability and Sparsity Assumptions

NeurIPS 2025 · Learning theory and algorithms › Matrix and tensor methods · approach: High-dimensional estimation theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes optimization-free closed-form estimators for the bivariate matrix-valued linear regression model Y_t = A X_t B + E_t (A row-stochastic nonnegative), with non-asymptotic bounds in dense and hard-thresholded sparse settings, validated on synthetic data and CIFAR-10 denoising.

Problem

Estimating the two parameter matrices A* (nonnegative, row-wise l1-normalized) and B* in the bilinear matrix-valued regression Y_t = A* X_t B* + E_t from matrix-valued observations, with finite-sample guarantees and identifiability.

Approach

Vectorize the model into a linear regression for C = (X^T X)^{-1} X^T M, whose entries equal A*_{ik} B*_{lj}; recover B* by summing over rows of A (using the l1 constraint) and A* by a ratio of entries (averaged over B-entries), then clip A to [0,1] and hard-threshold for sparsity. No optimization is needed.

Key results

• Empirical slope of error with respect to T is about -0.47 to -0.58 across norms, matching the predicted T^{-1/2} rate for both A-hat and B-hat.
• On CIFAR-10 with a controlled transformation, the corrected images are closer to originals than the noisy ones across noise factors epsilon (Frobenius distance D_o,c below D_o,n).
• Theory is proved under the ORT assumption X^T X = T I, and the paper states the experiments are run without ORT and still match the rates.

Measured effects reported in the paper

Surprising findings

• Error of A-hat increases with m but decreases with p and q, while B-hat shows the opposite trend, an asymmetry attributed to the normalization of A*.

Limitations

• Theory relies on the ORT assumption (X^T X = T I) and homoskedastic isotropic Gaussian noise.
• Requires the design to be a generating family (T >= mq) and assumes entries of A* strictly in (0,1) and sum of B* positive.
• Estimator of A is a ratio of random variables, analyzed only via a clipping workaround rather than a direct ratio analysis.

Experimental setup

Datasets: CIFAR-10 synthetic Gaussian/uniform simulations

Benchmarks: CIFAR-10 denoising (synthetic transformation)

Metrics: Frobenius norm error operator norm error max norm error empirical convergence slopes Frobenius distance of denoised images vs SNR

Baselines: KRO-PRO-FAC alpha-PCA ordinary least squares on vectorized model

Builds on: Kronecker product factorization matrix normal distribution trace regression

Closest papers by meaning

• Low-Rank-Sparse Subspace Representation for Robust Regression (CVPR 2017 · similarity 0.84)
• Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods (NeurIPS 2024 · similarity 0.84)
• Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem (ICLR 2024 · similarity 0.83)
• Spectral Estimation with Free Decompression (NeurIPS 2025 · similarity 0.83)
• Regularized least squares learning with heavy-tailed noise is minimax optimal (NeurIPS 2025 · similarity 0.83)
• Robust Sparse Regression with Non-Isotropic Designs (NeurIPS 2024 · similarity 0.82)