Research map / Learning theory and algorithms / Matrix and tensor methods
High-Dimensional Tensor Regression With Oracle Properties
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes nonconvex (SCAD/MCP) penalized least-squares tensor-on-tensor regression with entry-wise, fiber/slice-wise, and mode-wise/slice-wise low-rank penalties, proves oracle-rate error bounds, and solves it with an accelerated proximal gradient method.
Problem
High-dimensional tensor-on-tensor regression requires structural assumptions (sparsity or low-rankness) for identifiability, but convex penalties (Lasso, nuclear norm) introduce bias and yield slower rates than oracle estimators.
Approach
Apply decomposable nonconvex penalties (SCAD, MCP) to entry-wise, fiber-wise, slice-wise sparsity and to singular values of mode-wise or slice-wise unfoldings, prove the estimators attain the oracle statistical rate under RSC/RSM and minimum-signal conditions, and solve with a proximal gradient method with nonmonotone extrapolation.
Key results
- On synthetic 16x16x16 tensors, nonconvex entry-wise sparsity gives MSFE 0.4042 vs 0.6938 for convex (lower is better).
- Mode-wise low-rank nonconvex regularizer gives MSFE 0.5482 vs 1.7411 for convex on synthetic data.
- On ImageNet denoising (64x64x3, n=4000), nonconvex MSFE 134.59 vs convex 144.72 for entry-wise sparsity; mode-wise low-rank 35.55 vs 41.27.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a nonconvex SCAD penalty instead of a convex regularizer for entry-wise sparsity | Tensor regression · Synthetic data | MSFE: 0.6938 ± 0.0297 -> 0.4042 ± 0.0201 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for entry-wise sparsity | Tensor regression · Synthetic data | RMSE: 0.1004 ± 0.0023 -> 0.0992 ± 0.0021 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for mode-wise low-rankness | Tensor regression · Synthetic data | MSFE: 1.7411 ± 0.0953 -> 0.5482 ± 0.0395 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for mode-wise low-rankness | Tensor regression · Synthetic data | RMSE: 0.1096 ± 0.0020 -> 0.1002 ± 0.0012 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for entry-wise sparsity | Tensor regression · ImageNet 2012 image denoising | MSFE: 144.7160 ± 14.9947 -> 134.5864 ± 11.2950 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for entry-wise sparsity | Tensor regression · ImageNet 2012 image denoising | MPRE: 7.7498 ± 0.0457 -> 7.6072 ± 0.0301 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for mode-wise low-rankness | Tensor regression · ImageNet 2012 image denoising | MSFE: 41.2719 ± 3.5079 -> 35.5536 ± 1.4889 | Helped |
| Use a nonconvex SCAD penalty instead of a convex regularizer for mode-wise low-rankness | Tensor regression · ImageNet 2012 image denoising | MPRE: 1.1027 ± 0.0024 -> 1.0330 ± 0.0022 | Helped |
Limitations
- Only regularization-based structures are covered; tensor decomposition methods (CP, Tucker) are not theoretically analyzed.
- Theoretical guarantees rely on RSC/RSM, minimum signal strength, and sub-Gaussian assumptions that may be strong in practice.
- Experiments are limited to small synthetic tensors and a single image denoising setup.
Experimental setup
Datasets: synthetic Gaussian covariates ImageNet 2012
Benchmarks: ImageNet 2012 (image denoising subset)
Metrics: MSFE RMSE MPRE
Baselines: convex Lasso-type tensor regression (Raskutti et al. 2019) convex nuclear norm regularization
Builds on: Raskutti et al. 2019 high-dimensional tensor regression Yao et al. 2017 nonmonotone accelerated proximal gradient SCAD MCP
Closest papers by meaning
- Boosted Sparse and Low-Rank Tensor Regression (NeurIPS 2018 · similarity 0.90)
- Factor Augmented Tensor-on-Tensor Neural Networks (AAAI 2025 · similarity 0.89)
- Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression (ICML 2025 · similarity 0.88)
- Doubly Decomposing Nonparametric Tensor Regression (ICML 2016 · similarity 0.88)
- Generalized Tensor Decomposition for Understanding Multi-Output Regression under Combinatorial Shifts (NeurIPS 2024 · similarity 0.87)
- Efficient Nonparametric Tensor Decomposition for Binary and Count Data (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.