Research map / Vision-language and multimodal / Human and embodied modeling
MmCows: A Multimodal Dataset for Dairy Cattle Monitoring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MMCOWS, a two-week multimodal dataset (nine modalities including UWB, IMMU, CBT, ankle, multi-view RGB, environment, milk yield) from ten wearable-instrumented dairy cows, with 20,000 annotated frames for cow ID and behavior, and baseline benchmarks.
Problem
Existing dairy cattle datasets cover only one or two sensing modalities, so there is no multimodal dataset for comparing and combining sensors for precision livestock farming monitoring.
Approach
A synchronized multimodal sensing deployment (wearable neck tags with UWB/IMMU/pressure, implanted vaginal temperature loggers, ankle sensors, four isometric cameras, microclimate and weather sensors, milk records) on milking Holstein cows, with ground truth from multi-view visual annotation and an optimization-based 3D location method.
Key results
- Best behavior classifier UWB+HD+Akl reaches 0.737 average F1 under the temporal split, versus 0.141 for IMMU alone and 0.554 for single-view RGB.
- RGB multi-view fusion (RGBm) improves average F1 from 0.554 (RGBs) to 0.632 under the temporal split.
- Daily standing, lying, feeding, and drinking behaviors show significant correlations with THI over 13 days (e.g., drinking bout frequency r=0.802, p=0.001).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add head direction to UWB (ablation) | UWB behavior classification · Behavior classification OS | average F1: .707±.051 -> .707±.064 | No clear effect |
| Add head direction to UWB (ablation) | UWB behavior classification · Behavior classification TS | average F1: .717±.038 -> .737±.032 | Helped |
| Add ankle data to UWB and head direction (ablation) | UWB+HD behavior classification · Behavior classification OS | average F1: .707±.064 -> .714±.067 | No clear effect |
| Add ankle data to UWB and head direction (ablation) | UWB+HD behavior classification · Behavior classification TS | average F1: .737±.032 -> .737±.028 | No clear effect |
| Combine predictions from multiple camera views (ablation) | RGBs behavior classification · Behavior classification TS | average F1: .554±.077 -> .632±.075 | Helped |
| Use UWB instead of IMMU | Behavior classification · Behavior classification OS | average F1: .133±.060 -> .707±.051 | Helped |
Surprising findings
- IMMU-based behavior classification performs poorly (average F1 about 0.13-0.14), far below UWB and RGB, and fails on walking and several classes.
Limitations
- Ground-truth annotations cover only one day (20,000 frames) of the two-week recording.
- Walking is hard to distinguish from standing when frames are classified individually.
- Only ten cows carry wearable sensors, from a single barn over two weeks.
Experimental setup
Datasets: MMCOWS
Benchmarks: MMCOWS
Metrics: F1 score Pearson correlation coefficient p-value R2
Baselines: UWB-only random forest IMMU fully connected network RGBs single-view pipeline RGBm multi-view pipeline
Builds on: YOLOv8 EfficientNet
Closest papers by meaning
- Digital Twin-Driven Teat Localization and Shape Identification for Dairy Cow (Student Abstract) (AAAI 2024 · similarity 0.84)
- Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind (NeurIPS 2025 · similarity 0.83)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.82)
- AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark (NeurIPS 2025 · similarity 0.82)
- MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps (CVPR 2025 · similarity 0.81)
- HarvestNet: A Dataset for Detecting Smallholder Farming Activity Using Harvest Piles and Remote Sensing (AAAI 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.