Research map / Detection and segmentation / CNN features
SemStereo: Semantic-Constrained Stereo Matching Network for Remote Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemStereo, a semantic-guided cascade stereo network with selective disparity refinement and left-right semantic consistency loss for joint semantic segmentation and stereo matching on remote sensing imagery.
Problem
Semantic segmentation and stereo matching in remote sensing are treated separately or loosely coupled in multi-task networks, with the explicit connections between semantic categories and disparities left unmodeled.
Approach
Feeds deep semantically enriched shared features into a Fast-ACV cost volume (cascade instead of parallel structure), refines disparity with semantic-guided residuals (SSR), and enforces cross-view semantic consistency by warping the left semantic map with the disparity (LRSC).
Key results
- On US3D test, SemStereo reaches EPE 0.9582 and D1 4.58%, versus 1.1706 / 7.06% for Fast-ACVNet.
- On US3D semantic segmentation, SemStereo achieves 77.02% mIoU and 94.13% PA, above SegFormer (63.60% mIoU) and PSPNet (67.06%).
- Without semantic labels (SemStereo*), the model still reaches 0.2236 EPE and 0.731% D1 on WHU, the best among compared stereo methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SGC to the baseline (ablation) | SemStereo stereo network · US3D Jacksonville test | EPE (Pixel): 1.2087 -> 0.9995 | Helped |
| add SGC to the baseline (ablation) | SemStereo stereo network · US3D Jacksonville test | D1 (%): 7.28 -> 4.98 | Helped |
| add SSR to SGC-Net (ablation) | SemStereo stereo network · US3D Jacksonville test | EPE (Pixel): 0.9995 -> 0.9702 | Helped |
| add SSR to SGC-Net (ablation) | SemStereo stereo network · US3D Jacksonville test | D1 (%): 4.98 -> 4.76 | Helped |
| add LRSC to SGC-SSR-Net (ablation) | SemStereo stereo network · US3D Jacksonville test | EPE (Pixel): 0.9702 -> 0.9582 | Helped |
| add LRSC to SGC-SSR-Net (ablation) | SemStereo stereo network · US3D Jacksonville test | D1 (%): 4.76 -> 4.58 | Helped |
| add LRSC without explicit semantic label supervision (ablation) | SemStereo* · US3D Jacksonville test | EPE (Pixel): 1.0164 -> 0.9956 | Helped |
| add LRSC without explicit semantic label supervision (ablation) | SemStereo* · US3D Jacksonville test | D1 (%): 5.34 -> 5.00 | Helped |
| add explicit semantic label supervision (ablation) | SemStereo stereo network · US3D Jacksonville test | EPE (Pixel): 0.9956 -> 0.9582 | Helped |
| add explicit semantic label supervision (ablation) | SemStereo stereo network · US3D Jacksonville test | D1 (%): 5.00 -> 4.58 | Helped |
| add stereo matching supervision (ablation) | SemStereo semantic segmentation · US3D test | mIOU (%): 67.57 -> 77.02 | Helped |
Surprising findings
- Semantic labels from disparity-based LRSC without annotations (self-supervised via predicted semantics) still improve D1 by 6.4% in ablation.
Limitations
- Semantic segmentation and stereo gains are shown only on US3D and WHU, two remote sensing datasets.
- WHU lacks semantic labels, so the full model is evaluated only on US3D for joint tasks.
Experimental setup
Datasets: US3D WHU
Benchmarks: US3D WHU
Metrics: EPE D1 mIoU PA
Baselines: StereoNet S2Net S3Net DSMNet HMSMNet GwcNet PSMNet IGEV-Stereo ACVNet Fast-ACVNet FCN-8s UNet
Builds on: Fast-ACV MobileViTv2 DSMNet S2Net S3Net
Closest papers by meaning
- Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation (AAAI 2025 · similarity 0.87)
- SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations (CVPR 2025 · similarity 0.86)
- SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery (NeurIPS 2024 · similarity 0.86)
- SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation (CVPR 2024 · similarity 0.86)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.86)
- Large Spatial Model: End-to-end Unposed Images to Semantic 3D (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.