Research map / Video / Video segmentation and tracking
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
Jennifer J. Sun, Markus Marks, Andrew Wesley Ulmer, Dipam Chakraborty, Brian Geuther, Edward Hayes, Heng Jia, Vivek Kumar, Sebastian Oleszko, Zachary Partridge, Milan Peelman, Alice Robie and 11 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretraining on Kinetics400 (rhoBYOL) instead of training on animal data (ablation) | rhoBYOL · Mouse Triplet and Ant Beetle manual behaviors | F1 manual behaviors: 0.2688 vs 0.1832 (mouse); 0.6967 vs 0.6178 (beetle) | Improved |
| Pretraining on Kinetics400 for MAE and MaskFeat (ablation) | MAE, MaskFeat · Mouse and beetle downstream tasks | MSE and F1 across tasks | Hurt |
| Cube (spatiotemporal random) masking instead of frame or tube masking in MAE (ablation) | MAE · Mouse Triplet | Time of Day MSE / Manual Behaviors F1: cube .0816 vs tube .0835 vs frame .0886 (time of day); cube .2309 vs tube .2249 vs frame .2020 (behaviors) | Mixed |
| Cube masking instead of frame or tube masking in MAE (ablation) | MAE · Ant Beetle | Duration MSE / Manual Behaviors F1: cube .0283 vs frame .0301, tube .0285 (duration); cube .6444 vs frame .6169, tube .5802 (behaviors) | Improved |
| Increasing number of positives in rhoBYOL from 2 to 4 (or 3 for beetle) (ablation) | rhoBYOL · Mouse Triplet and Ant Beetle | F1 / MSE across tasks: mouse best at 2 or 4 depending on task; beetle best at 3 positives | Mixed |
| Adding hand-crafted keypoint features and SimCLR to pretrained BEiT (community method) | Mouse video challenge method · Mouse Triplet | Exp. Day MSE and Strain F1: Exp. Day .0093 vs .0102 for MAE | Improved |
| Trajectory-only methods instead of video for mice | Trajectory-BERT · Mouse Triplet | Strain F1: .7202 vs 1.0000 for MAE | Hurt |
| Perceiver masked modeling on trajectories vs GPT-style next-frame prediction for flies | Fly group trajectory methods · Fly Group manual behaviors | F1 manual behaviors: Trajectory-GPT .246 vs Trajectory-Perceiver .197 on manual behaviors; Perceiver better on fly type and female vs male | Mixed |
| Community hand-crafted features and pretrained-model features versus pure self-supervised video models | Community methods on manual behaviors · Mouse and beetle manual behaviors | F1 manual behaviors: Vision Ensemble .2750 mouse; Multimodal MoCo/SimCLR .7299 beetle vs best SSL .2688 / .6967 | Improved |
Closest papers by meaning
- Relax, it doesn’t matter how you get there: A new self-supervised approach for multi-timescale behavior analysis (NeurIPS 2023 · similarity 0.91)
- The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions (NeurIPS 2021 · similarity 0.90)
- Learning Disentangled Behavior Embeddings (NeurIPS 2021 · similarity 0.89)
- Agent-to-Sim: Learning Interactive Behavior Models from Casual Longitudinal Videos (ICLR 2025 · similarity 0.87)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.87)
- Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
