Research map / Learning theory and algorithms / Matrix and tensor methods

Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor

NeurIPS 2024 · Learning theory and algorithms › Matrix and tensor methods · approach: Low-rank tensor completion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows the block trifocal tensor of calibrated/uncalibrated cameras has an explicit Tucker factorization with multilinear rank (6,4,4) independent of camera count, and uses HOSVD with hard thresholding (HOSVD-HT) to jointly recover scales and camera poses for synchronization.

Problem

Recovering global camera poses (locations and orientations) from the block trifocal tensor of three-view geometry, where each trifocal tensor block is known only up to an unknown scale and many blocks are missing.

Approach

The block trifocal tensor factors as a Tucker product with a fixed 6x4x4 core and camera-dependent factor matrices, giving a low multilinear rank (6,4,4) regardless of the number of cameras; this rank constraint fixes the unknown block scales and enables a tensor-completion-style HOSVD synchronization.

Key results

• Block trifocal tensor has multilinear rank (6,4,4) independent of number of cameras when cameras are not collinear and scales are fixed.
• Proposed method achieves the best location estimation on 4 of 6 EPFL datasets and competitive translation errors on 8 of 14 Photo Tourism datasets.
• Rotation estimation is superior on only 2 of 6 EPFL datasets, and it reaches reasonable rotation estimates on 10 of 14 Photo Tourism datasets, below LUD.

Measured effects reported in the paper

Surprising findings

• The method fails on CastleP19 and CastleP30 because sparse triplet graphs prevent high completion rates.
• The algorithm works well only when the signs of the unknown scales are mostly uniform; the synthetic experiment manually corrected signs using ground truth.

Limitations

• Rotation estimates are weaker than location estimates and below LUD on many datasets.
• Performance depends on the density of trifocal tensor estimates; sparse viewing graphs (e.g., CastleP19/30) fail.
• Cubic scaling in number of cameras limits scalability to large datasets.

Experimental setup

Datasets: EPFL datasets Photo Tourism datasets

Benchmarks: EPFL Photo Tourism

Metrics: mean translation error median translation error rotation error

Baselines: NRFM LUD BATA MPLS

Builds on: HARD-IMPUTE Tyler's M-estimator GlueStick STE NRFM LUD BATA

Closest papers by meaning

• QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition (CVPR 2026 · similarity 0.92)
• Pose Synchronization Under Multiple Pair-Wise Relative Poses (CVPR 2023 · similarity 0.88)
• Cycle-Sync: Robust Global Camera Pose Estimation through Enhanced Cycle-Consistent Synchronization (NeurIPS 2025 · similarity 0.86)
• Improving Robustness for Joint Optimization of Camera Pose and Decomposed Low-Rank Tensorial Radiance Fields (AAAI 2024 · similarity 0.85)
• Performance Gaps in Multi-view Clustering under the Nested Matrix-Tensor Model (ICLR 2024 · similarity 0.83)
• Efficient Detection of Long Consistent Cycles and its Application to Distributed Synchronization (CVPR 2024 · similarity 0.83)