Research map / Architectures and efficiency / RNNs and normalization
SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SVDinsTN, a tensor network paradigm that inserts diagonal factors on every edge of a fully-connected TN and solves a regularized (l1 + Tikhonov) PAM/ADMM model so TN structure is searched jointly with the cores, avoiding repeated sampling-evaluation.
Problem
Tensor network structure search (TN-SS, topology and rank) is NP-hard, and state-of-the-art sampling-evaluation methods require evaluating many candidate structures, each with an iterative core optimization, at prohibitive cost.
Approach
Insert diagonal factors between every pair of TN cores in a fully-connected TN so that factor sparsity reveals a compact structure; optimize cores and diagonal factors jointly under an l1 sparsity regularizer via a PAM-based algorithm, with an SVD-based initialization that bounds the initial TN ranks.
Key results
- On light field data at RE bound 0.05, SVDinsTN reaches comparable CR to TN-SS methods (e.g., 6.92% on Bunny vs 4.52% for TNALE) with about 100-1000x lower run time (0.029k s vs 18.36k s on Bunny).
- Structure revealing success rate is over 95% in all 100-trial synthetic tests on 4th- and 5th-order tensors.
- In video tensor completion at 90% missing ratio, SVDinsTN achieves the highest MPSNR on all four videos (e.g., 32.401 on Bunny vs 30.729 for TW).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVDinsTN instead of TNLS | Light-field tensor representation · Bunny, Knights, and Truck, RE bound 0.05 | run time (×1000s): 63.70 -> 0.029; 75.51 -> 0.104; 119.8 -> 0.152 | Helped |
| Use SVDinsTN instead of TNLS | Light-field tensor representation · Bunny, Knights, and Truck, RE bound 0.05 | compression ratio: 4.26% -> 6.92%; 4.73% -> 5.64%; 4.99% -> 6.42% | Hurt |
| Use SVDinsTN instead of the strongest listed baseline | Tensor completion · Bunny video, 90% missing | MPSNR: 30.729 -> 32.401 | Helped |
| Use SVDinsTN instead of the strongest listed baseline | Tensor completion · News video, 90% missing | MPSNR: 30.027 -> 31.643 | Helped |
| Use SVDinsTN instead of the strongest listed baseline | Tensor completion · Salesman video, 90% missing | MPSNR: 30.685 -> 31.684 | Helped |
| Use SVDinsTN instead of the strongest listed baseline | Tensor completion · Silent video, 90% missing | MPSNR: 31.731 -> 32.706 | Helped |
| Replace the proposed initialization with random initialization (ablation) | SVDinsTN · Truck, RE bound 0.05 | compression ratio: 6.42% -> 8.17% | Hurt |
| Replace the proposed initialization with random initialization (ablation) | SVDinsTN · Truck, RE bound 0.05 | run time (×1000s): 0.152 -> 0.474 | Hurt |
Surprising findings
- The repeated sampling-evaluation TN-SS baselines are far slower than even pre-defined-topology methods, with SVDinsTN requiring only a single optimization run.
Limitations
- No theoretical guarantee that the recovered TN structure is optimal (acknowledged as open).
- Tensor sizes and TN orders tested are modest; higher-order results limited to synthetic data with RE bound 0.01.
- Regularization parameter gamma must be tuned per experiment (selected from an interval).
Experimental setup
Datasets: Stanford light field data (Bunny, Knights, Truck) color videos (Bunny, News, Salesman, Silent) synthetic tensors
Benchmarks: Stanford light field (Bunny, Knights, Truck) color videos Bunny/News/Salesman/Silent tensor completion synthetic 4th/5th-order structure recovery 6th/8th/10th-order synthetic tensors
Metrics: compression ratio run time success rate relative error MPSNR
Baselines: TRALS FCTNALS TNGreedy TNGA TNLS TNALE FBCP TMac TMacTT TRLRF TW
Builds on: FCTN decomposition proximal alternating minimization SVD ADMM
Closest papers by meaning
- tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) (ICML 2024 · similarity 0.89)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.86)
- Deep Tree Tensor Networks (NeurIPS 2025 · similarity 0.86)
- Attention-based Iterative Decomposition for Tensor Product Representation (ICLR 2024 · similarity 0.86)
- Fast Tensor Completion via Approximate Richardson Iteration (ICML 2025 · similarity 0.85)
- OTLRM: Orthogonal Learning-based Low-Rank Metric for Multi-Dimensional Inverse Problems (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.