Research map / Video / Video segmentation and tracking
Low-Latency Neural Stereo Streaming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLSS, a parallel low-latency neural stereo video codec that replaces sequential inter-view motion compensation with a bidirectional feature shifting module (BiShiftMod) inside parallel autoencoders.
Problem
Stereo video compression methods compress left and right views sequentially, limiting parallelization, runtime and latency for applications like VR and autonomous driving.
Approach
Replace sequential disparity/motion compensation between views with a learned bidirectional horizontal shift module (with groupwise and concatenation correlations) that lets parallel left/right autoencoders and hyper-codecs share information, reducing cross-view mutual information in latents.
Key results
- 50.6% BD-rate savings vs MV-HEVC on Cityscapes (LSVC: 32.7%).
- 18.2% and 15.8% BD-rate savings vs MV-HEVC on KITTI 2012 and KITTI 2015 respectively.
- Transmitter is 1.7x faster and receiver 1.9x faster than LSVC at 512x512; LSVC needs about 2.8x more FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LLSS instead of LSVC | stereo video compression · CityScapes test | BD-rate (%) vs MV-HEVC: -32.7 -> -50.6 | Helped |
| use LLSS instead of LSVC | stereo video compression · KITTI 2012 test | BD-rate (%) vs MV-HEVC: -17.1 -> -18.2 | Helped |
| use LLSS instead of LSVC | stereo video compression · KITTI 2015 test | BD-rate (%) vs MV-HEVC: -13.4 -> -15.8 | Helped |
| add BiShiftMod to the codec (ablation) | LLSS · CityScapes test | BD-rate: 7.3% | Helped |
| add BiShiftMod to the hypercodec (ablation) | LLSS · CityScapes test | BD-rate: 6.1% | Helped |
| add BiShiftMod to the codec and hypercodec (ablation) | LLSS · CityScapes test | BD-rate: 13.0% | Helped |
| add GroupCor to BiShiftMod (ablation) | LLSS · CityScapes test | BD-rate: 3.7% | Helped |
| add CatCor to BiShiftMod (ablation) | LLSS · CityScapes test | BD-rate: 4.3% | Helped |
| use parallel left and right feature streams instead of a single shared feature (ablation) | LLSS · CityScapes test | BD-rate saving: 62% | Helped |
Surprising findings
- Removing the parallel design (concatenating left and right features into a single shared feature) increases BD-rate substantially, with the parallel feature streams giving 62% BD-rate saving over the single shared feature.
Limitations
- KITTI results obtained without fine-tuning, and the gap to LSVC narrows substantially on KITTI 2012 and 2015.
- DCVC was evaluated without fine-tuning since its training code was not released, so comparison is less reliable.
Experimental setup
Datasets: Vimeo-90K Cityscapes
Benchmarks: Cityscapes KITTI 2012 KITTI 2015
Metrics: BD-rate PSNR MS-SSIM bits-per-pixel FLOPs MACs inference time
Baselines: H.265 MV-HEVC FVC DCVC LSVC
Builds on: FVC LSVC DCVC SASIC
Closest papers by meaning
- 3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors (ICML 2025 · similarity 0.85)
- Towards Practical Real-Time Neural Video Compression (CVPR 2025 · similarity 0.84)
- CAMSIC: Content-aware Masked Image Modeling Transformer for Stereo Image Compression (AAAI 2025 · similarity 0.83)
- Augmented Deep Contexts for Spatially Embedded Video Coding (CVPR 2025 · similarity 0.82)
- C3: High-Performance and Low-Complexity Neural Compression from a Single Image or Video (CVPR 2024 · similarity 0.82)
- DeNC: Unleash Neural Codecs in Video Streaming with Diffusion Enhancement (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.