Research map / 3D vision / Gaussian splatting
3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3D-LMVIC, a learned multi-view image codec that uses 3D Gaussian Splatting depth priors for depth-based disparity/overlap masks, a depth map compression model, and distance-based view ordering.
Problem
Multi-view image compression relies on 2D projection similarity to estimate disparities, which fails for complex, wide-baseline multi-camera disparities such as those in VR and autonomous driving.
Approach
Use 3D Gaussian Splatting to render per-view depth maps, derive pixel-wise disparities and occlusion/overlap masks from camera geometry, compress depth maps jointly with images using cross-view depth prediction, and order views with a proven matrix-based distance metric.
Key results
- On Tanks&Temples, 3D-LMVIC achieves BDBR of -47.48% (PSNR) and -63.69% (MS-SSIM) relative to MV-HEVC, beating BiSIC (-30.89%, -49.96%).
- In alignment experiments on Tanks&Temples Train scene, the proposed 3D-GS-based alignment reaches 18.14 PSNR and 0.8053 MS-SSIM, above optical flow and homography baselines.
- Removing the masks increases bitrate by 7.19% (PSNR) and 8.47% (MS-SSIM); random view ordering increases bitrate by 42.4% (PSNR) versus sorted ordering.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add cross-view information (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 41.07% | Helped |
| Add cross-view information (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 44.24% | Helped |
| Align reference features before fusion (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 42.75% | Helped |
| Align reference features before fusion (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 47.52% | Helped |
| Use image and depth masks (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 7.19% | Helped |
| Use image and depth masks (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 8.47% | Helped |
| Add depth prediction to depth map compression (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 8.03% | Helped |
| Add depth prediction to depth map compression (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 8.02% | Helped |
| Order views using the proposed sequence-ordering method instead of random ordering (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 42.4% | Helped |
| Order views using the proposed sequence-ordering method instead of random ordering (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 50.64% | Helped |
| Use proposed sequence ordering instead of manual sorting (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at PSNR: 3.76% | Hurt |
| Use proposed sequence ordering instead of manual sorting (ablation) | 3D-LMVIC · Tanks&Temples | Bitrate at MS-SSIM: 2.97% | Hurt |
Surprising findings
- HAC, a 3D Gaussian compression method, has very high BDBR (e.g., 636.81% PSNR on Tanks&Temples) relative to MV-HEVC.
Limitations
- Encoder MACs (479.43G) are high relative to some baselines, and total parameter count is larger (41.92M encoder).
- Requires 3D-GS training per scene and camera calibration via SfM, adding preprocessing cost.
- HAC baseline bpp computed by training 3DGS on test data, making comparison less direct.
Experimental setup
Datasets: Tanks&Temples Mip-NeRF 360 Deep Blending
Benchmarks: Tanks&Temples Mip-NeRF 360 Deep Blending
Metrics: PSNR MS-SSIM BDBR bpp MACs runtime
Baselines: MV-HEVC HESIC HESIC+ MASIC SASIC LDMIC LDMIC-Fast BiSIC BiSIC-Fast HAC HEVC
Builds on: 3D Gaussian Splatting hyperprior entropy model quadtree partition-based entropy model LDMIC BiSIC
Closest papers by meaning
- ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS (NeurIPS 2025 · similarity 0.87)
- CAT-3DGS: A Context-Adaptive Triplane Approach to Rate-Distortion-Optimized 3DGS Compression (ICLR 2025 · similarity 0.87)
- A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding (NeurIPS 2024 · similarity 0.86)
- 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video (CVPR 2025 · similarity 0.86)
- Efficient Decoupled Feature 3D Gaussian Splatting via Hierarchical Compression (CVPR 2025 · similarity 0.86)
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.