Research map / Video / Video segmentation and tracking
Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes E-MAC, a density-embedded masked autoencoder with spatial adaptive masking and optical-flow temporal fusion for video object counting, and releases the DroneBird bird video counting dataset.
Problem
Video object counting suffers from fore-background imbalance due to sparse targets, causing under- and over-prediction; existing video counting datasets are human-centric and there is no large video bird counting dataset.
Approach
Treats the density map as an auxiliary modality in a masked autoencoder (DEMO), uses density-guided spatial adaptive masking (SAM) that keeps foreground tokens with a background retention probability, and fuses adjacent frames via optical-flow warping and cross-attention residuals (TCF).
Key results
- On DroneBird, E-MAC reaches MAE 38.72 and RMSE 42.92, versus 39.11/50.08 for the best image baseline MAN.
- On FDST, E-MAC achieves MAE 1.29 and RMSE 1.69, improving over the runner-up STGN (1.38/1.82).
- Ablation on FDST: removing DEMO, SAM, or TCF each increases MAE; full model reaches 1.29 versus 2.45 for the baseline ViT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use E-MAC instead of the strongest baseline, FRVCC | video object counting · Mall | MAE: 1.41 -> 1.35 | Helped |
| use E-MAC instead of the strongest baseline, STGN | video object counting · FDST | MAE: 1.38 -> 1.29 | Helped |
| use E-MAC instead of the strongest baseline, PET | video object counting · VSCrowd | MAE: 6.6 -> 6.0 | Helped |
| use E-MAC instead of the strongest baseline, MAN | video object counting · DroneBird | MAE: 39.11 -> 38.72 | Helped |
| add TCF to the pure transformer baseline (ablation) | FDST counting · FDST | MAE: 2.45 -> 2.32 | Helped |
| add SAM to TCF (ablation) | FDST counting · FDST | MAE: 2.32 -> 1.57 | Helped |
| add DEMO to TCF (ablation) | FDST counting · FDST | MAE: 2.32 -> 1.69 | Helped |
| combine DEMO and SAM with TCF (ablation) | FDST counting · FDST | MAE: 1.57 -> 1.29 | Helped |
| add L_TV to L_fuse (ablation) | E-MAC · FDST | MAE: 1.87 -> 1.80 | Helped |
| add L_opt to L_fuse and L_TV (ablation) | E-MAC · FDST | MAE: 1.80 -> 1.60 | Helped |
| add L_cur to L_fuse and L_TV (ablation) | E-MAC · FDST | MAE: 1.80 -> 1.39 | Helped |
| add L_cur and L_opt to L_fuse and L_TV (ablation) | E-MAC · FDST | MAE: 1.80 -> 1.29 | Helped |
Surprising findings
- A model trained on VSCrowd transfers poorly to DroneBird (MAE 183.31), indicating a large gap between crowd and bird data.
Limitations
- Image size and temporal integration increase computational cost, so 320x320 input was chosen as a trade-off.
- DroneBird is limited to 50 videos from drones at 30-60 m altitude in a few habitats.
Experimental setup
Datasets: FDST Mall VSCrowd DroneBird
Benchmarks: Mall FDST VSCrowd DroneBird
Metrics: MAE RMSE
Baselines: MCNN CSRNet CAN MAN HMoDE PET Gramformer EPF PFTF GNANet FRVCC STGN
Builds on: MAE MultiMAE PWCNet CSRNet PET
Closest papers by meaning
- Open-World Object Counting in Videos (AAAI 2026 · similarity 0.85)
- Adapting Lightweight Image-based Counting Models for Video Crowd Counting (CVPR 2026 · similarity 0.85)
- Extending Video Masked Autoencoders to 128 frames (NeurIPS 2024 · similarity 0.85)
- EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens (ICML 2024 · similarity 0.85)
- VideoMAC: Video Masked Autoencoders Meet ConvNets (CVPR 2024 · similarity 0.84)
- A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.