Research map / Large language models / AI and society
Position: Tensor Networks are a Valuable Asset for Green AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that tensor networks (CP, Tucker, TT) are a valuable tool for Green AI, supported by a review of Green AI efficiency metrics and literature examples from kernel machines and deep learning.
Problem
The exponentially growing compute demand of AI research harms economic, social and environmental sustainability and inclusivity, and the field lacks standardized efficiency reporting; the paper asks whether tensor networks can help address this.
Approach
Links tensor networks to Green AI for the first time, arguing their logarithmic compression (exponential-to-linear dependence on tensor order D) makes them a model-development strategy for efficiency, and evaluates literature TN examples using Green AI efficiency metrics.
Key results
- Kim et al. (compression of CNNs with TNs) report up to 86.4% memory reduction, 79.7% FLOPS reduction, 72.8% runtime reduction and 76.6% electricity reduction with top-5 accuracy loss of at most 2.2%.
- Novikov et al. (exponential machines) report a 21.3x speed-up on a classification task at a cost of 11.46% accuracy relative to a competing method.
- Abronin et al. compress GPT-2 small with a 1.5x compression ratio at a relative performance loss of 30.83%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply tensor-network compression | Kernel machine · synthetic classification | accuracy: $-\,11.46\,\%$; $21.3 \times$ speed-up | Mixed |
| Apply tensor-network compression | Kernel machine · Abalone regression | accuracy: $-\,4.39\,\%$; $1.6 \times $ speed-up | Mixed |
| Apply tensor-network compression | Kernel machine · Airline regression | accuracy: $+\,1.13\,\%$; $2.5 \times$ speed-up | Helped |
| Apply tensor-network compression | AlexNet · ImageNet classification | accuracy: $+\,2.17\,\%$; $1.8 \times$ speed-up | Helped |
| Apply tensor-network compression | LSTM · UCF11 action recognition | accuracy: $+\,1.13\,\%$; $14.0 \times$ speed-up | Helped |
| Apply tensor-network compression | ResNet · CIFAR-100 classification | accuracy: $+\,3.25\,\%$; $2.4 \times$ compression ratio | Helped |
| Apply tensor-network compression | GPT-2 Small · Wikitext-2 NLP | accuracy: $-\,30.83\,\%$; $1.5 \times$ compression ratio | Hurt |
| Apply tensor networks to convolutional layers | CNN · multiple datasets | memory: 86.4% reduction | Helped |
| Apply tensor networks to convolutional layers | CNN · multiple datasets | top-5 accuracy: reduced by no more than 2.2% | Mixed |
| Apply tensor networks to recurrent layers | LSTM · UCF11 action recognition | accuracy: increase accuracy by 15.6%; speed up convergence by 14 times | Helped |
| Compress CNN weights with tensor networks | CNN · multiple image classification datasets | accuracy: slightly increase the accuracy; compression ratio between $ 2.4 \times$ and $8.3 \times$ | Helped |
Surprising findings
- Tucker decomposition still scales exponentially in D (O(R^D)), unlike CP and TT, which scale linearly in D.
- Embodied emissions can make up the largest share of AI emissions even when operational emissions are reduced; over 50% of Meta's emissions are attributed to embodied costs.
Limitations
- No new experiments; evidence is drawn from a small selection of literature examples, and few TN papers report efficiency metrics.
- TN benefits may be limited for weakly correlated data, and optimal rank selection lacks theoretical guarantees (NP-hard rank determination).
- Potential rebound effects and system-level impacts of efficiency gains are acknowledged but not quantified.
Experimental setup
Datasets: Abalone Airline ImageNet UCF11 CIFAR-100 Wikitext-2
Benchmarks: Abalone Airline ImageNet UCF11 CIFAR-100 Wikitext-2
Metrics: speed-up compression ratio relative change in accuracy FLOPS parameter count electricity usage memory runtime
Builds on: Green AI (Schwartz et al.) tensor train decomposition Novikov et al. tensorizing neural networks
Closest papers by meaning
- tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) (ICML 2024 · similarity 0.86)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.85)
- The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones for Progress Towards More Efficient Model Inference (ICML 2025 · similarity 0.85)
- Activation Map Compression through Tensor Decomposition for Deep Learning (NeurIPS 2024 · similarity 0.85)
- Benchmarking Predictive Coding Networks -- Made Simple (ICLR 2025 · similarity 0.85)
- SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.