Research map / Architectures and efficiency / RNNs and normalization
Matrix-Free Shared Intrinsics Bundle Adjustment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a matrix-free bundle adjustment solver exploiting shared-intrinsics structure, with single-precision stabilization tricks, achieving large speedups and memory savings over Ceres on SfM datasets.
Problem
Bundle adjustment is usually accelerated for photo collections where each image has its own camera; shared intrinsics bundle adjustment (SI-BA), common in practice, is under-optimized, and single-precision BA is numerically unstable.
Approach
Exploits SI-BA structure with a flat state/connectivity representation and computes augmented-Hessian products on the fly (matrix-free) instead of storing Hpl and Hlp, plus a compact per-image scaling replacing Jacobi scaling and a pseudo-inverse with a condition-number bound for low-parallax landmarks in single precision.
Key results
- CPU-f64 gives roughly 4x, CPU-f32 roughly 8x, and GPU-f32 roughly 20x speedup over Ceres at tolerances 0.1 and 0.001.
- Peak memory is about an order of magnitude lower than Ceres on most datasets (e.g. EuRoC 1182 MB vs 73 MB for CPU-f32 in fixed intrinsics; actually reported as 1192 vs 71 MB), but not on LaMAR with dynamic intrinsics (8715 MB Ceres vs 1652 MB CPU-f32, still lower but reduced advantage).
- Single-precision solvers converge nearly identically to double-precision Ceres per iteration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · Tanks & Temples, Aachen, EuRoC, and KITTI; dynamic intrinsics, τ = 0.1 | time to reach tolerance: Ceres: 3.5, 10.8, 5.2, 6.9 s; CPU-f64: 0.4, 2.1, 0.4, 0.9 s | Helped |
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · LaMAR (LIN), dynamic intrinsics, τ = 0.1 | time to reach tolerance: 17.1 -> 12.1 s | Helped |
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · LaMAR (LIN), dynamic intrinsics, τ = 0.1 | time to reach tolerance: 17.1 -> 6.8 s | Helped |
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · LaMAR (LIN), dynamic intrinsics, τ = 0.1 | time to reach tolerance: 17.1 -> 1.4 s | Helped |
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · Tanks & Temples, Aachen, EuRoC, and KITTI; dynamic intrinsics | peak memory: Ceres: 1102, 4333, 1303, 2548 MB; CPU-f64: 125, 571, 100, 300 MB | Helped |
| Use matrix-free shared-intrinsics BA instead of Ceres | Bundle adjustment · LaMAR (LIN), dynamic intrinsics | peak memory: 8715 -> 3022 MB | Helped |
| Use single precision instead of double precision | Matrix-free CPU solver · Tanks & Temples, Aachen, EuRoC, KITTI, and LaMAR; dynamic intrinsics | peak memory: 125 -> 81; 571 -> 358; 100 -> 74; 300 -> 192; 3022 -> 1652 MB | Helped |
| Use single precision instead of double precision | Matrix-free CPU solver · Bundle adjustment convergence | cost reduction per iteration | No clear effect |
| Sort landmarks by observation count, breaking ties by minimum observing image ID (ablation) | Matrix-free products Hpl and Hlp · EuRoC (V1-01) | computation time (ms): Random: 1.39, 8.57; Count & Idx: 1.19, 1.88 | Helped |
| Use the proposed image-specific translation scaling instead of no variable scaling (ablation) | Single-precision BA · EuRoC (V1-01) | condition number | Helped |
| Disconnect near-zero-depth observations whose single- and double-precision projections differ by more than 0.005 pixels (ablation) | Single-precision BA preprocessing · Bundle adjustment | numerical stability: 0.005 pixels | Helped |
| Use a condition-bound-triggered pseudo-inverse for ill-conditioned landmark blocks (ablation) | Single-precision BA · Low-parallax landmarks | numerical stability | Helped |
Surprising findings
- Matrix-free products are faster than precomputed sparse matrix products, even in double precision.
- Ceres's Jacobian-based approach to H products is slower than explicit normal-equation products, yet the reported 4x double-precision speedup exceeds what memory-throughput analysis alone suggests.
Limitations
- Dense storage of Hcl makes runtime and memory advantage shrink on LaMAR with dynamic intrinsics across 20 cameras.
- GPU-f32 implementation lacks floating-point error accumulation mitigations and cannot reach tolerances below 1e-4 on KITTI.
- Evaluation uses a single laptop test bench and fixed 50 PCG iterations.
Experimental setup
Datasets: Tanks & Temples (Courthouse) EuRoC (V1-01) Aachen (Day-Night) KITTI (00) LaMAR (LIN)
Benchmarks: Tanks & Temples (Courthouse) EuRoC (V1-01) Aachen (Day-Night) KITTI (00) LaMAR (LIN)
Metrics: runtime to reach cost tolerance peak memory consumption (MB) cost convergence per iteration
Baselines: Ceres Apero
Builds on: Ceres Solver implicit Schur complement Jacobi preconditioning (Wu et al. 2011 style BA) Levenberg-Marquardt
Closest papers by meaning
- A Consensus-Based Framework for Distributed Bundle Adjustment (CVPR 2016 · similarity 0.87)
- Square Root Bundle Adjustment for Large-Scale Reconstruction (CVPR 2021 · similarity 0.85)
- BA-T: An Iterative Transformer for Two-View Bundle Adjustment (NeurIPS 2026 · similarity 0.84)
- From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers (CVPR 2024 · similarity 0.80)
- Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor (NeurIPS 2024 · similarity 0.80)
- BADGR: Bundle Adjustment Diffusion Conditioned by Gradients for Wide-Baseline Floor Plan Reconstruction (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.