Research map / 3D vision / Face and shape
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAM 3D Animal, a promptable set-prediction network that reconstructs multiple animals from one image into SMAL+ meshes using keypoint/mask prompts, trained with a new synthetic multi-animal dataset Herd3D.
Problem
Multi-animal 3D mesh reconstruction from single in-the-wild images, where existing methods focus on single animals, use pre-cropped inputs, and lack multi-instance 3D training data.
Approach
A SAM-style promptable transformer decoder predicts up to 30 SMAL+ hypotheses jointly with DETR-style Hungarian matching (no NMS or bounding box input), accepts keypoint and mask prompts, and is trained with a synthetic multi-animal dataset (Herd3D) generated by adapting GenZoo.
Key results
- Prompt-free variant reaches 80.7 mm PA-MPJPE on Animal3D vs 81.0 for AniMer, and 87.9 PCK@0.1 on APTv2 vs 64.1 for GenZoo.
- With ground-truth keypoint prompts, APTv2 PCK@0.1 reaches 89.0 and AP 57.4; Animal Kingdom AP reaches 60.1 and mAP 17.7 vs AniMer's 10.4.
- Keypoint prompt count improves performance monotonically from 0 to 15 keypoints (Animal3D PA-MPJPE 80.8 to 77.4).
Surprising findings
- Removing mask prompts leaves performance almost unchanged, while removing keypoint prompts reverts to the unprompted baseline.
- The prompt-free model has lower AP than GenZoo on APTv2 (49.4 vs 55.5) despite higher PCK.
Limitations
- Restricted to quadruped-like animals by the SMAL+ shape space.
- Relative depth ordering between animals is not explicitly constrained, causing errors under severe occlusion.
- Inference is slower than cropped single-animal baselines.
Experimental setup
Datasets: Animal Pose APTv2 AwA-Pose Stanford Extra Animal3D Herd3D Animal Kingdom
Benchmarks: Animal3D APTv2 Animal Kingdom
Metrics: PA-MPJPE PCK@0.1 AP mAP
Baselines: 3D Fauna GenZoo AniMer
Builds on: SAM 3D Body SMAL+ GenZoo DETR DN-DETR
Closest papers by meaning
- Reconstructing Animals and the Wild (CVPR 2025 · similarity 0.90)
- Web-Scale Collection of Video Data for 4D Animal Reconstruction (NeurIPS 2025 · similarity 0.89)
- BEAST3D: animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting (NeurIPS 2026 · similarity 0.89)
- Learning the 3D Fauna of the Web (CVPR 2024 · similarity 0.89)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.86)
- Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.