Research map / Video / Video segmentation and tracking
Multi-View Dynamic Reflection Prior for Video Glass Surface Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VGSD-Net, the first video glass surface detector, which uses multi-view dynamic reflection cues via location-aware and context-enhanced modules, and releases the VGSD-D video glass dataset (19,166 frames, 297 videos).
Problem
Glass surface detection in dynamic video scenes, where image-based methods fail and no video glass dataset exists.
Approach
Exploit reflections that change location and appearance as the camera moves: a Location-aware Reflection Extraction module uses masked deformable attention with a position map from the predicted glass mask, and a Context-enhanced Reflection Integration module fuses features across frames temporally and spatially with contrast-enhanced context and transformer blocks.
Key results
- VGSD-Net achieves IoU 0.802, accuracy 0.899, BER 9.54 and MAE 0.099 on VGSD-D, the best among 17 compared methods.
- Outperforms the best single-image glass detector GlassNet (IoU 0.762) by about 20.63% relative IoU per the paper's text.
- Ablation shows the reflection prior is key: removing the reflection loss raises BER from 9.54 to 12.04; the model runs at 15.04 FPS with 64.06M parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VGSD-Net with multi-view dynamic reflection cues | VGSD-Net · VGSD-D test | IoU: 0.765 -> 0.802 | Helped |
| add reflection cues to the encoder-and-mask-head baseline (ablation) | VGSD-Net · VGSD-D test | IoU: 0.734 -> 0.802 | Helped |
| use an explicit position map in masked deformable attention instead of implicit mask concatenation (ablation) | VGSD-Net LRE · VGSD-D test | IoU: 0.743 -> 0.802 | Helped |
| add LRE with explicit position-aware reflection extraction (ablation) | VGSD-Net · VGSD-D test | IoU: 0.751 -> 0.802 | Helped |
| add CFA to LRE (ablation) | VGSD-Net · VGSD-D test | IoU: 0.751 -> 0.774 | Helped |
| add TTB after LRE and CFA (ablation) | VGSD-Net · VGSD-D test | IoU: 0.774 -> 0.794 | Helped |
| add STB after LRE and CFA (ablation) | VGSD-Net · VGSD-D test | IoU: 0.774 -> 0.786 | Helped |
| use both TTB and STB after LRE and CFA (ablation) | VGSD-Net · VGSD-D test | IoU: 0.794 -> 0.802; 0.786 -> 0.802 | Helped |
| add intermediate mask supervision (ablation) | VGSD-Net · VGSD-D test | IoU: 0.781 -> 0.802 | Helped |
| add auxiliary reflection supervision (ablation) | VGSD-Net · VGSD-D test | IoU: 0.753 -> 0.802 | Helped |
| add the reflection-region penalty term Lpen (ablation) | VGSD-Net · VGSD-D test | IoU: 0.783 -> 0.802 | Helped |
Surprising findings
- SAM misclassifies objects behind glass as non-glass due to reflection and refraction, performing worse than the proposed method by a large margin.
Limitations
- Fails when temporal reflections are too weak across all frames, e.g., windows behind the main glass are classified as non-glass.
- Relies on reflection cues, so glass regions with little reflection are harder to detect.
Experimental setup
Datasets: VGSD-D DAVIS VOS
Benchmarks: VGSD-D
Metrics: IoU pixel accuracy BER MAE
Baselines: GateNet MINet ZoomNet UFO DeepLab Segformer SAM TVSD SC-Cor MirrorNet PMDNet VCNet
Builds on: deformable attention vision transformer masked deformable attention (Zhu et al. 2023)
Closest papers by meaning
- MVGD-Net: A Novel Motion-aware Video Glass Surface Detection Method (AAAI 2026 · similarity 0.94)
- Rich Context Aggregation With Reflection Prior for Glass Surface Detection (CVPR 2021 · similarity 0.91)
- Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection (AAAI 2025 · similarity 0.87)
- Effective Video Mirror Detection with Inconsistent Motion Cues (CVPR 2024 · similarity 0.84)
- Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation (CVPR 2025 · similarity 0.83)
- ZOOM: Learning Video Mirror Detection with Extremely-Weak Supervision (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.