Research map / Large language models / Agents, code and math
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes tnGPS, an LLM prompting pipeline (knowledge categorization, recombination, incremental innovation, diversity injection) that iteratively generates new tensor network structure search sampling algorithms, which outperform TNGA, TNLS, GREEDY and TnALE on image compression and TGP model compression.
Problem
Tensor network structure search (TN-SS) requires selecting TN ranks and topology, and existing hand-crafted sampling-based heuristics balance exploration and exploitation poorly and get stuck in local minima.
Approach
Model human-expert research workflow (idea pool, clustering, idea dropout, recombination, incremental innovation, diversity injection) as an LLM prompting loop that writes and iteratively refines Python implementations of the GenerateSamples operator, with a sandbox to validate generated code and evaluation feedback guiding the next iteration.
Key results
- On BSD500 test images, Ho-2 reached a log compression ratio of 1.352 with RSE 0.130, the best among TNGA, TNLS, GREEDY, TnALE and Ho-1/Ho-3 (1.332 best baseline).
- Ablation objective on the image task: tnGPS reached 0.1102 versus 0.1558 for the best baseline.
- On TGP model compression with random initialization, only Ho-2 matched the TT-initialized structures, and on MG+random it was the only discovered algorithm that beat the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove knowledge recombination (KR) phase from tnGPS (ablation) | tnGPS on MG model compression · MG TGP compression (objective) | objective value (lower better): 0.1102 -> 0.1308 without KR | Helped |
| Remove incremental innovation (II) phase from tnGPS (ablation) | tnGPS on MG model compression · MG TGP compression (objective) | objective value (lower better): 0.1102 -> 0.1273 without II | Helped |
| Remove diversity injection (DI) phase from tnGPS (ablation) | tnGPS on MG model compression · MG TGP compression (objective) | objective value (lower better): 0.1102 -> 0.1239 without DI | Helped |
| Use GPT-4 instead of GPT-3.5/Claude models as the LLM (ablation) | tnGPS · image 2 train (LLM ablation) | objective value (lower better): GPT-4 0.1813 vs GPT-3.5 0.1842, Claude-1 0.1840, Claude-2 0.1834 | Helped |
| Remove 8 of 12 interface description comments (ablation) | tnGPS · image 2 train (LLM ablation) | objective value (lower better): 0.1813 -> 0.1819 | No clear effect |
| Discover algorithms with tnGPS (LLM-driven evolution of GenerateSamples) versus existing TN-SS algorithms | TN-SS for natural image compression · BSD500 grayscale 256x256 images, test set | log compression ratio (higher better): Ho-2 1.352 best vs TNGA 1.332; Ho-1 1.329 and Ho-3 1.322 below best baselines | Mixed |
| Discover algorithms with tnGPS versus existing methods | TGP model compression, random initialization · CCPP and MG (out-of-domain) | number of parameters (lower better): Ho-2 2.24 (CCPP) and 3.01 (MG) vs TNGA 2.36 / 12.69; Ho-1 and Ho-3 worse on MG (6.81, 27.74) | Mixed |
| Discover algorithms with tnGPS versus existing methods | TGP model compression, TT initialization · CCPP and MG (out-of-domain) | number of parameters (lower better): All algorithms reach 2.24 (CCPP) / 3.01 (MG) or 3.36 baseline; tnGPS variants only differ in samples to first find struc | No clear effect |
Surprising findings
- Discovered algorithms are no longer Markovian, using information from all historical samples, and one uses an inverted annealing schedule that increases mutation rate over time.
- Less capable LLMs (GPT-3.5, Claude-1/2) gave weaker results, and the incomplete interface description ablation barely changed performance.
Limitations
- Performance of discovered algorithms varies with the underlying LLM, so results depend on model choice.
- Prompts were tuned by trial and error and may need manual adjustment for different LLMs.
- Evaluation uses few training images (4) and small experiments with a single run reported.
Experimental setup
Datasets: BSD500 CCPP MG
Benchmarks: BSD500 image compression (10 test images) CCPP tensorial Gaussian process compression MG tensorial Gaussian process compression
Metrics: log compression ratio relative squared error number of parameters objective function value samples to first find structure
Baselines: TNGA TNLS GREEDY TnALE Tensor train baseline
Builds on: TNGA TNLS GREEDY TnALE FunSearch-style LLM evolution
Closest papers by meaning
- SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective (CVPR 2024 · similarity 0.89)
- Position: Tensor Networks are a Valuable Asset for Green AI (ICML 2024 · similarity 0.86)
- Deep Tree Tensor Networks (NeurIPS 2025 · similarity 0.85)
- S2MVTC: a Simple yet Efficient Scalable Multi-View Tensor Clustering (CVPR 2024 · similarity 0.85)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.85)
- Geometry-aware training of factorized layers in tensor Tucker format (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.