Research map / Video / Video segmentation and tracking
MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MammAlps, a multimodal, multi-view camera-trap video dataset of wild mammal behavior from the Swiss Alps, with two benchmarks (hierarchical multimodal clip recognition and long-term multi-view event understanding) and a token-merging baseline.
Problem
Wildlife behavior monitoring from camera traps is limited by the lack of densely annotated, multimodal, multi-view fieldwork video datasets that support modern video understanding models.
Approach
A fieldwork camera-trap dataset with hierarchical (activity and action) individual-level behavior labels, paired audio and reference scene segmentation maps, and multi-view event-level labels (activities, species, group size, weather), plus an offline spatio-temporal token merging method over a VideoMAE/ViT-Base backbone for long multi-view events.
Key results
- Video-only VideoMAE reaches 0.487 species, 0.428 activity, 0.458 action mAP in joint prediction (B1).
- Adding audio to video (V+A) raises B1 average mAP from 0.453 to 0.472.
- Token-merging baseline reaches average per-class mAP of 0.498 on B2 with ToMe reduction factor r=11.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio to video (ablation) | VideoMAE on B1 · B1 | overall mAP: 0.453 -> 0.472 | Helped |
| add segmentation maps to video and audio (ablation) | VideoMAE on B1 · B1 | overall mAP: 0.472 -> 0.438 | Hurt |
| add segmentation maps to audio-only input (ablation) | VideoMAE on B1 · B1 | overall mAP: 0.184 -> 0.270 | Helped |
| jointly predict species, activities, and actions rather than each task separately (ablation) | VideoMAE on B1 · B1 species | mAP: 0.537 -> 0.487 | Hurt |
| jointly predict species, activities, and actions rather than each task separately (ablation) | VideoMAE on B1 · B1 activities | mAP: 0.440 -> 0.428 | Hurt |
| jointly predict species, activities, and actions rather than each task separately (ablation) | VideoMAE on B1 · B1 actions | mAP: 0.447 -> 0.458 | Helped |
| use r=11 rather than r=14 (ablation) | Joint event understanding model on B2 · B2 | overall mAP: 0.476 -> 0.498 | Helped |
| use r=11 rather than r=14 (ablation) | Joint event understanding model on B2 · B2 species | mAP: 0.343 -> 0.439 | Helped |
| use r=11 rather than r=14 (ablation) | Joint event understanding model on B2 · B2 activities | mAP: 0.483 -> 0.450 | Hurt |
| use r=11 rather than r=14 (ablation) | Joint event understanding model on B2 · B2 meteorological conditions | mAP: 0.653 -> 0.634 | Hurt |
| use r=11 rather than r=14 (ablation) | Joint event understanding model on B2 · B2 individual count | mAP: 0.478 -> 0.593 | Helped |
| add CamID, elapsed-time, and source embeddings (ablation) | B2 individual count model with r=14 · B2 individual count | mAP: 0.514|0.505 -> 0.592|0.478 | Helped |
Surprising findings
- Adding segmentation maps to video did not improve B1 performance, although they helped the audio-only model (+0.059 mAP).
- Multi-task joint prediction outperformed single-task models for behavior recognition on B1.
Limitations
- Only five species from one National Park at one elevation over six weeks, limiting generalization.
- B2 test set has only 86 events, insufficient for rare classes.
- Behavior labels are subjective and prone to annotator bias, especially social behaviors.
Experimental setup
Datasets: MammAlps
Benchmarks: MammAlps B1 (behavior recognition) MammAlps B2 (event understanding)
Metrics: mAP
Baselines: VideoMAE AudioMAE ToMe
Builds on: VideoMAE ToMe AudioMAE ByteTrack MegaDetector
Closest papers by meaning
- The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition (CVPR 2025 · similarity 0.90)
- BuckTales: A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes (NeurIPS 2024 · similarity 0.89)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.87)
- BirdCollect: A Comprehensive Benchmark for Analyzing Dense Bird Flock Attributes (AAAI 2024 · similarity 0.86)
- MOVEBENCH: A Benchmark for Global-Scale Wildlife Movement Forecasting (NeurIPS 2026 · similarity 0.86)
- Generating the Wild: Individual-Consistent Image-to-Video Generation for Wildlife (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.