Research map / Video / Video segmentation and tracking
RipVIS: Rip Currents Video Instance Segmentation Benchmark for Beach Monitoring and Safety
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RipVIS, a 184-video rip current video instance segmentation benchmark, benchmarks Mask R-CNN, Cascade Mask R-CNN, SparseInst and YOLO11 with a post-hoc Temporal Confidence Aggregation (TCA) step.
Problem
Rip current segmentation in beach videos is hard due to amorphous shapes and lack of large, expert-annotated, instance-segmentation video data.
Approach
A large, geographically diverse video instance segmentation dataset with train/val/test splits for rip currents, plus a training-free temporal post-processing (TCA) that aggregates per-instance confidence heatmaps across frames via tracking, smoothing and hysteresis thresholding.
Key results
- SparseInst PVTv2 achieves the highest F2 of 0.751 on the test split without TCA; with TCA it reaches 0.780.
- Cascade Mask R-CNN F2 rises from 0.648 to 0.670 with TCA; Mask R-CNN from 0.593 to 0.625.
- YOLO11 variants reach high precision (up to 0.812 for YOLO11l) but lower recall (about 0.59-0.63), while TCA lowers FPS substantially (e.g., YOLO11n 128.20 to 34.48).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Temporal Confidence Aggregation (TCA) (ablation) | Mask-RCNN · RipVIS test split | F2: 0.593 -> 0.625 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | Cascade Mask-RCNN · RipVIS test split | F2: 0.648 -> 0.670 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | YOLO11n · RipVIS test split | F2: 0.583 -> 0.613 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | YOLO11s · RipVIS test split | F2: 0.636 -> 0.666 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | YOLO11m · RipVIS test split | F2: 0.644 -> 0.665 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | YOLO11l · RipVIS test split | F2: 0.622 -> 0.646 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | YOLO11x · RipVIS test split | F2: 0.632 -> 0.664 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | SparseInst R-50 · RipVIS test split | F2: 0.710 -> 0.749 | Helped |
| Add Temporal Confidence Aggregation (TCA) (ablation) | SparseInst PVTv2 · RipVIS test split | F2: 0.751 -> 0.780 | Helped |
Surprising findings
- TCA lowers inference FPS substantially for one-stage YOLO11 models (e.g., 128.2 to 34.5 for YOLO11n).
- Adding automatically generated annotations for non-manually annotated frames slightly decreased most metrics.
Limitations
- Performance remains modest across all models, showing the task is hard; TCA can increase false positives under fast camera motion.
- Optimal TCA settings differ between stationary and moving cameras, so no single configuration fits all videos.
- Dataset excludes flash or traveling rip currents.
Experimental setup
Datasets: RipVIS
Benchmarks: RipVIS
Metrics: IoU mAP AP50 precision recall F1 F2 FPS
Baselines: Mask R-CNN Cascade Mask R-CNN SparseInst R-50 SparseInst PVTv2 YOLO11n YOLO11s YOLO11m YOLO11l YOLO11x
Builds on: Mask R-CNN Cascade Mask R-CNN SparseInst YOLO11 Canny edge detector
Closest papers by meaning
- LAqua: Laplacian Pyramids for Aquatic Segmentation (Student Abstract) (AAAI 2025 · similarity 0.82)
- CoastalBench: A Decade-Long High-Resolution Dataset to Emulate Complex Coastal Processes (ICML 2025 · similarity 0.82)
- SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey (NeurIPS 2024 · similarity 0.82)
- Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset (ICML 2024 · similarity 0.81)
- A Unified Image-Dense Annotation Generation Model for Underwater Scenes (CVPR 2025 · similarity 0.81)
- ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition (NeurIPS 2026 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.