Research map / Learning theory and algorithms / Matrix and tensor methods

Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression

ICML 2025 · Learning theory and algorithms › Matrix and tensor methods · approach: Tensor network decompositions

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoRT, a transfer learning framework for tensor regression that fuses source and target tasks with a TNN-based fusion regularizer followed by target-only refinement, plus a distributed variant D-LoRT, evaluated on synthetic compressed sensing and video tensor completion.

Problem

Tensor regression suffers from limited target sample size, and transfer from source tasks is complicated by model shift, covariate shift, and decentralized data.

Approach

A tubal-nuclear-norm fusion regularizer that encourages low tubal rank in the target parameter and in target-source differences, followed by a target-specific TNN-regularized refinement step; a distributed variant aggregates locally estimated source models to reduce communication.

Key results

• On akiyo video tensor completion, LoRT Step2 PSNR rises from 22.19 dB at K=1 to 40.32 dB at K=8, while TNN baseline stays around 14 dB.
• In synthetic compressed sensing, LoRT error decreases as the number of source tasks grows, whereas TNN and k-Sup stay flat.
• D-LoRT outperforms the TNN baseline on video completion but remains below centralized LoRT (e.g., 27.37 dB vs 40.32 dB on akiyo at K=8).

Measured effects reported in the paper

Surprising findings

• At K=1 on akiyo, D-LoRT (25.71 dB) beats centralized LoRT Step1 (22.15 dB), though it is overtaken as K grows.

Limitations

• Theory assumes i.i.d. Gaussian designs and sampling within tasks.
• Experiments use simulated or controlled settings and lack real-world data without ground truth.
• TNN proximal operator is computationally expensive (per-slice SVDs), limiting scalability.

Experimental setup

Datasets: YUV video (akiyo, bridge, grandma, hall) UCF-101 (Apply Eye Make-up, Blowing Candles) synthetic Gaussian tensor regression data

Benchmarks: synthetic tensor compressed sensing YUV video tensor completion

Metrics: relative error PSNR

Baselines: TNN k-Sup D-LoRT (distributed variant) Convex Tucker OITNN

Builds on: TransFusion t-SVD LoRA

Closest papers by meaning

• High-Dimensional Tensor Regression With Oracle Properties (ICML 2025 · similarity 0.88)
• Factor Augmented Tensor-on-Tensor Neural Networks (AAAI 2025 · similarity 0.87)
• Generalized Tensor Decomposition for Understanding Multi-Output Regression under Combinatorial Shifts (NeurIPS 2024 · similarity 0.86)
• Deep Rank-One Tensor Functional Factorization for Multi-Dimensional Data Recovery (AAAI 2025 · similarity 0.86)
• Performance Gaps in Multi-view Clustering under the Nested Matrix-Tensor Model (ICLR 2024 · similarity 0.86)
• Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.85)