Research map / Video / Video segmentation and tracking
ZOOM: Learning Video Mirror Detection with Extremely-Weak Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns video mirror detection from only per-frame zero-one mirror-presence indicators, using CAM-based temporal fusion and a temporal similarity-contrast module, and introduces a new video mirror dataset.
Problem
Mirror detection normally requires costly pixel-wise labels, and fully-supervised detectors over-segment mirrors in frames that contain none. The paper asks whether mirror segmentation can be learned from only frame-level mirror presence indicators.
Approach
Uses the temporal existence of mirrors as weak supervision: CAM localization maps are fused across adjacent mirror frames and a non-mirror frame, and a temporal similarity-contrast module enforces feature similarity between adjacent mirror regions and contrast against distant non-mirror frames.
Key results
- On the proposed dataset's frames with mirrors, ZOOM reaches IoU 0.513 and F-beta 0.774, competitive with fully-supervised detectors; on frames without mirrors it achieves TNR 0.994 and ER 0.091, the best among compared methods.
- On the VMD dataset, ZOOM obtains IoU 0.294 and F-beta 0.448, trailing fully-supervised methods such as VMDNet (IoU 0.567).
- Ablation: removing the SE block drops IoU/Fβ from 0.513/0.774 to 0.433/0.682.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use extremely weak frame-level 0/1 mirror indicators for video mirror detection | ZOOM · Proposed dataset, frames without mirrors | TNR: 0.983 -> 0.994 | Helped |
| Use extremely weak frame-level 0/1 mirror indicators for video mirror detection | ZOOM · VMD dataset | IoU: 0.567 -> 0.294 | Hurt |
| Add temporal fusion to CAM localization (ablation) | ZOOM · Proposed dataset test set | IoU: 0.276 -> 0.334 | Helped |
| Add the feedback constraint to temporal fusion (ablation) | ZOOM · Proposed dataset test set | IoU: 0.334 -> 0.473 | Helped |
| Add the proposed feedback-constrained temporal CAM localization (ablation) | ZOOM · Proposed dataset test set | IoU: 0.276 -> 0.473 | Helped |
| Add temporal frame information instead of processing a single frame (ablation) | ZOOM · Proposed dataset test set | IoU: 0.318 -> 0.513 | Helped |
| Add adjacent-frame similarity modeling (ablation) | ZOOM · Proposed dataset test set | IoU: 0.318 -> 0.422 | Helped |
| Add distant-frame contrast modeling (ablation) | ZOOM · Proposed dataset test set | IoU: 0.318 -> 0.392 | Helped |
| Add both adjacent-frame similarity and distant-frame contrast modeling (ablation) | ZOOM · Proposed dataset test set | IoU: 0.422 -> 0.513 | Helped |
| Add feature similarity loss Lfs (ablation) | ZOOM · Proposed dataset test set | IoU: 0.474 -> 0.513 | Helped |
| Add feature contrast loss Lfc (ablation) | ZOOM · Proposed dataset test set | IoU: 0.492 -> 0.513 | Helped |
| Replace projection loss with Lovász-Softmax loss (ablation) | ZOOM · Proposed dataset test set | Fβ: 0.724 -> 0.774 | Helped |
Surprising findings
- Video-based VMDNet generalizes worse than image-based detectors on the new dataset, which the authors attribute to its reliance on temporal appearance correspondence.
- ZOOM trained with only zero-one indicators is competitive with fully-supervised detectors on mirror frames, but is clearly weaker on the VMD dataset.
Limitations
- Fails when mirrors are small or far away across the whole video.
- Misclassifies mirror presence when scenes contain mirror-like glossy objects.
- Pixel-level accuracy is not always precise, and it trails fully-supervised methods on the VMD dataset.
Experimental setup
Datasets: Charades Charades-Ego VMD dataset (Lin, Tan, and Lau 2023)
Benchmarks: ZOOM video mirror dataset VMD dataset
Metrics: IoU F-beta (beta=0.3) MAE TNR ER
Baselines: MirrorNet PMDNet SANet VCNet SAT HetNet VMDNet
Builds on: CAM Lovász-Softmax loss ASPP SE block
Closest papers by meaning
- Weakly-Supervised Mirror Detection via Scribble Annotations (AAAI 2024 · similarity 0.93)
- Progressive Mirror Detection (CVPR 2020 · similarity 0.92)
- Learning To Detect Mirrors From Videos via Dual Correspondences (CVPR 2023 · similarity 0.91)
- Detect Any Mirrors: Boosting Learning Reliability on Large-Scale Unlabeled Data with an Iterative Data Engine (CVPR 2025 · similarity 0.91)
- Effective Video Mirror Detection with Inconsistent Motion Cues (CVPR 2024 · similarity 0.89)
- MIRROR: Make Your Object-Level Multi-View Generation More Consistent with Training-Free Rectification (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.