Research map / Video / Video segmentation and tracking
The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PanAf-FGBG, a wild chimpanzee camera-trap dataset of paired foreground and background videos, and uses it to quantify background reliance and propose latent-space background subtraction for out-of-distribution behaviour recognition.
Problem
Behaviour recognition from camera trap video relies on background shortcuts, and the effect of background information and background duration on out-of-distribution generalisation has not been systematically studied because no dataset pairs foreground and background footage from the same cameras.
Approach
Pair every chimpanzee video with a temporally close background-only video from the same camera, provide overlapping and disjoint camera-location splits to measure in- vs out-of-distribution behaviour, and subtract weighted background features in latent space during training to improve OOD recognition.
Key results
- Latent-space background subtraction with linear scheduling improves 3D R50 disjoint mAP by +5.42% and MViT-V2 disjoint mAP by +3.75%.
- Background-only models reach roughly 65% or more of foreground-trained performance (background-to-foreground mAP ratio never below 0.65 across architectures).
- MViT-V2 shows the lowest background reliance, with a background-to-foreground uAP ratio on disjoint data ~30% and ~13% lower than 2D R50 and 3D R50.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on background-only videos instead of foreground videos | 2D R50 · PanAf-FGBG overlap | mAP: 41.28 -> 30.03 | Hurt |
| Train on background-only videos instead of foreground videos | 2D R50 · PanAf-FGBG disjoint | mAP: 18.08 -> 18.27 | No clear effect |
| Late-fuse a background stream with the foreground stream | 3D R50 · PanAf-FGBG disjoint | mAP: +8.28% | Helped |
| Late-fuse a background stream with the foreground stream | MViT-V2 · PanAf-FGBG overlap | uAP: 75.44 -> 72.82 | Hurt |
| Subtract real background in input space (ablation) | 2D R50 · PanAf-FGBG disjoint | mAP: +7.76% | Helped |
| Subtract synthetic background in input space (ablation) | 2D R50 · PanAf-FGBG disjoint | mAP: 18.08 -> 26.65 | Helped |
| Apply latent-space subtraction with no schedule (ablation) | 2D R50 · PanAf-FGBG disjoint | mAP: 18.08 -> 17.94 | No clear effect |
| Apply latent-space subtraction with linear schedule (ablation) | 2D R50 · PanAf-FGBG disjoint | mAP: 18.08 -> 17.17 | Hurt |
| Apply latent-space subtraction with exponential schedule (ablation) | 2D R50 · PanAf-FGBG disjoint | mAP: 18.08 -> 17.73 | No clear effect |
| Subtract real background in input space (ablation) | 3D R50 · PanAf-FGBG disjoint | mAP: 30.89 -> 25.05 | Hurt |
| Subtract synthetic background in input space (ablation) | 3D R50 · PanAf-FGBG disjoint | mAP: 30.89 -> 27.98 | Hurt |
| Apply latent-space subtraction with no schedule (ablation) | 3D R50 · PanAf-FGBG disjoint | mAP: 30.89 -> 34.85 | Helped |
Surprising findings
- Background-only models trained on synthetic (SAM2-masked) backgrounds outperform those trained on real backgrounds, yet real backgrounds dominate when used for subtraction.
- Input-space background subtraction helps 2D R50 but does not improve 3D R50 or MViT-V2, while embedding-space subtraction helps the latter.
Limitations
- Restricted to statically installed camera trap videos
- Latent-space subtraction requires additional data and compute for paired backgrounds
- Concatenating background frames may create unnatural artifacts
Experimental setup
Datasets: PanAf-FGBG Kinetics-400
Benchmarks: PanAf-FGBG overlapping split PanAf-FGBG disjoint split
Metrics: uAP mAP background-to-foreground AP ratio
Baselines: Dummy classifier 2D ResNet-50 3D ResNet-50 MViT-V2 Dual-stream F+B Input-space background subtraction
Builds on: SlowFast/ResNet-50 action recognition MViT-V2 SAM2 segmentation with mean-pixel filling Wilds distribution shift benchmark PanAf20k
Closest papers by meaning
- MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps (CVPR 2025 · similarity 0.90)
- BuckTales: A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes (NeurIPS 2024 · similarity 0.88)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.86)
- Generating the Wild: Individual-Consistent Image-to-Video Generation for Wildlife (NeurIPS 2026 · similarity 0.85)
- Low-power Continuous Remote Behavioral Localization with Event Cameras (CVPR 2024 · similarity 0.85)
- BirdCollect: A Comprehensive Benchmark for Analyzing Dense Bird Flock Attributes (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.