Research map / 3D vision / Face and shape
Coarse-To-Fine Tensor Trains for Compact Visual Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PuTT, a coarse-to-fine tensor-train (QTT) representation learned by iteratively upsampling via a prolongation MPO with TT-SVD rank truncation, improving compression, denoising, and missing-data completion for image, 3D, and NeRF fitting.
Problem
Tensor-train representations of visual data are highly compact but hard to optimize with gradient methods, which get stuck in local minima and struggle with noisy or incomplete data.
Approach
Learn a quantized tensor train (QTT) coarse-to-fine: fit at a low resolution, then apply a fixed bond-dimension-3 linear-interpolation prolongation MPO directly in TT format, truncate ranks with TT-SVD, and continue training at finer resolution.
Key results
- At 16k 2D image resolution PuTT outperforms baselines by over 2.5 PSNR and 0.1 SSIM across compression ratios.
- On NeRF Synthetic-NeRF/NSVF/TanksTemples, a 7MB PuTT model outperforms small baselines by over 1 PSNR on NSVF (35.58 vs 34.57 for TensoRF VM-30 no shrinking).
- Training on 1% of pixels, PSNR rises from 1.87 without upsampling to 28.73 with seven upsampling steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use seven upsampling steps instead of no upsampling | PuTT on 1% training data · 2D image fitting with 1% of input data | PSNR: 1.87 -> 28.73 | Helped |
| Use seven upsampling steps instead of no upsampling | PuTT on 1% training data · 2D image fitting with 1% of input data | SSIM: 0.0018 -> 0.7349 | Helped |
| Use PuTT instead of small TensoRF baselines | Small novel-view-synthesis models · NSVF | PSNR: over one PSNR | Helped |
| Use PuTT instead of TensoRF | Novel-view-synthesis models · Blender Chair, near test views | PSNR: 35.59 -> 37.12 | Helped |
| Use PuTT instead of TensoRF | Novel-view-synthesis models · Blender Ficus, far test views | PSNR: 30.16 -> 31.06 | Helped |
| Use PuTT instead of TensoRF | Novel-view-synthesis models · Blender all scenes, all test views | PSNR: 30.43 -> 31.66 | Helped |
| Add coarse-to-fine upsampling | PuTT trained on noisy visual data · Noise removal with noise levels exceeding 0.2 σ | SSIM: more than 0.1 | Helped |
| Add four upsampling steps and train for 1024 iterations rather than train without upsampling for 16k iterations (ablation) | PuTT · 2D image fitting | PSNR | Helped |
| Add four upsampling steps and train for 1024 iterations rather than train without upsampling for 16k iterations (ablation) | PuTT · 2D image fitting | SSIM | Helped |
| Add upsampling (ablation) | QTT on Lena 512x512x3 with σ=0.05 · Lena 512x512x3 | PSNR: 34.958 ± 0.083 -> 36.100 ± 0.002 | Helped |
| Add upsampling (ablation) | QTT on Lena 512x512x3 with σ=0.5 · Lena 512x512x3 | PSNR: 12.320 ± 1.610 -> 36.103 ± 0.006 | Helped |
| Vary initialization standard deviation from 0.001 to 0.5 (ablation) | PuTT · Lena 512x512x3 | PSNR: 36.103 ± 0.001 -> 36.103 ± 0.006 | No clear effect |
Surprising findings
- Without upsampling, QTT training is highly sensitive to initialization std (PSNR drops to 12.32 at std 0.5), while PuTT stays near 36.1 across all tested stds.
Limitations
- Advantage over CP/Tucker/VM diminishes at lower resolutions (e.g., 1k images) and less compressed settings.
- TT-SVD baseline does not apply to optimization-based settings such as missing data or NeRF.
- Rank growth from upsampling requires TT-SVD truncation at each step.
Experimental setup
Datasets: Girl With a Pearl Earring Tokyo gigapixel Marseille Pluto Westerlund Flower John Hopkins Turbulence Blender (Synthetic-NeRF) NSVF TanksTemples
Benchmarks: Synthetic-NeRF NSVF TanksTemples Girl With a Pearl Earring 16k Flower 3D John Hopkins Turbulence 1024^3
Metrics: PSNR SSIM model size (MB) compression ratio
Baselines: CP Tucker VM TT-SVD TensoRF TT-NF NeRF Instant-NGP Plenoxels DVGO K-Planes PlenOctrees
Builds on: TensoRF TT-NF TT-SVD quantized tensor train Instant-NGP
Closest papers by meaning
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.87)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.87)
- Tensor Decomposition Based Memory-Efficient Incremental Learning (ICML 2025 · similarity 0.86)
- High-Order Contrastive Learning with Fine-grained Comparative Levels for Sparse Ordinal Tensor Completion (ICML 2024 · similarity 0.85)
- Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression (ICML 2025 · similarity 0.85)
- SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.