Research map / Architectures and efficiency / RNNs and normalization
Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MultiHeadDepth (multi-head cost volume with LayerNorm-dot-product cosine approximation) and HomoDepth (adds homography prediction head with rectification positional encoding to skip stereo rectification) for low-latency stereo depth on AR glasses.
Problem
Stereo depth estimation for AR glasses needs low latency, but rectification preprocessing and the non-ML cost volume computation dominate latency, and unstable extrinsics make rectification unreliable.
Approach
Replace cosine similarity in the cost volume with LayerNorm plus group-wise dot product followed by pointwise convolution (multi-head), and eliminate online rectification by predicting a homography with a multi-task head and encoding it as a 2D rectification positional encoding inside the cost volume.
Key results
- MultiHeadDepth gives 11.8-30.3% accuracy improvement and 22.9-25.2% latency reduction versus ARGOS on laptop CPU/GPU.
- HomoDepth reduces end-to-end latency by 44.5% (abstract) versus preprocessing-based pipelines while handling unrectified inputs; on DTU df AbsRel 0.087 vs 0.183 for ARGOS.
- INT8 quantization reduces latency 7.7-43.4% with a 3.1-14.7% accuracy drop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the ARGOS cost volume with multi-head cost volume | ARGOS / MultiHeadDepth · SceneFlow | AbsRel: 0.102 -> 0.091 | Helped |
| replace the ARGOS cost volume with multi-head cost volume | ARGOS / MultiHeadDepth · SceneFlow | D1: 0.23 -> 0.35 | Hurt |
| replace the ARGOS cost volume with multi-head cost volume | ARGOS / MultiHeadDepth · ADT | AbsRel: 0.133 -> 0.094 | Helped |
| replace the ARGOS cost volume with multi-head cost volume | ARGOS / MultiHeadDepth · DTU | AbsRel: 0.122 -> 0.101 | Helped |
| add homography prediction and rectification positional encoding, eliminating preprocessing (ablation) | MultiHeadDepth / HomoDepth · DTU | AbsRel: 0.099 -> 0.098 | Helped |
| add homography prediction and rectification positional encoding, eliminating preprocessing (ablation) | MultiHeadDepth / HomoDepth · DTU df | AbsRel: 0.107 -> 0.087 | Helped |
| add homography prediction and rectification positional encoding, eliminating preprocessing (ablation) | MultiHeadDepth / HomoDepth · SceneFlow persp | AbsRel: 0.114 -> 0.097 | Helped |
| eliminate rectification preprocessing (ablation) | MultiHeadDepth / HomoDepth · DTU | latency (ms): 884.4 -> 761.2 | Helped |
Surprising findings
- Post-training INT8 quantization yields only modest accuracy loss while cutting latency substantially.
Limitations
- Latency measured on laptop, Jetson Orin Nano, and Snapdragon proxies rather than actual AR glasses hardware.
- ADT evaluation restricted to 145 of 236 scenarios without humans; DTU df is a custom synthetic-scaled variant.
- Homography assumes d_l/d_r approx 1, i.e. planar central plane or far scenes.
Experimental setup
Datasets: SceneFlow DTU Robot Image Dataset Aria Digital Twin (ADT) Middlebury 2014
Benchmarks: SceneFlow Middlebury 2014 ADT DTU DTU df SceneFlow persp
Metrics: AbsRel D1 RMSE latency (ms)
Baselines: ARGOS MobileStereoNet-2D MobileStereoNet-3D Dynamic-Stereo Selective-Stereo
Builds on: ARGOS MVSNet Multi-head attention Homoscedastic uncertainty multi-task loss
Closest papers by meaning
- Rectification-specific Supervision and Constrained Estimator for Online Stereo Rectification (CVPR 2025 · similarity 0.86)
- PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching (NeurIPS 2025 · similarity 0.85)
- UniDepth: Universal Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.85)
- MVSAnywhere: Zero-Shot Multi-View Stereo (CVPR 2025 · similarity 0.85)
- MonSter: Marry Monodepth to Stereo Unleashes Power (CVPR 2025 · similarity 0.85)
- QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.