Research map / 3D vision / Face and shape
Learning the 3D Fauna of the Web
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a pan-category deformable 3D model of over 100 quadruped animal species jointly from single-view Internet images, using a Semantic Bank of Skinned Models for feed-forward single-image reconstruction.
Problem
Reconstructing articulated, textured 3D models of many animal species from single-view Internet images, where rare species have too few images for category-specific models to generalize.
Approach
A learned memory bank of key-value pairs queries DINO features to produce semantically adaptive base shapes shared across species, combined with a mask discriminator using random viewpoints to counter frontal photographer bias.
Key results
- On PASCAL VOC horses, KT-PCK@0.1 of 0.539 versus 0.429 for MagicPony and 0.329 for A-CSM.
- PCK@0.1 of 0.782 on APT-36K, 0.841 on PASCAL VOC keypoints, and 0.901 on Animal3D, outperforming MagicPony and A-CSM on all datasets reported.
- Reconstructs an articulated mesh from a single image in seconds, versus 10-20 minutes of optimization for LASSIE, Hi-LASSIE and Zero-1-to-3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3D-Fauna instead of existing baselines | Animal reconstruction · PASCAL VOC | KT-PCK@0.1: 0.429 -> 0.539 | Helped |
| Use 3D-Fauna instead of existing baselines | Animal reconstruction · PASCAL VOC | PCK@0.1: 0.687 -> 0.782 | Helped |
| Use 3D-Fauna instead of existing baselines | Animal reconstruction · APT-36K | PCK@0.1: 0.756 -> 0.841 | Helped |
| Use 3D-Fauna instead of existing baselines | Animal reconstruction · Animal3D | PCK@0.1: 0.867 -> 0.901 | Helped |
| Add a Semantic Bank of Skinned Models (ablation) | 3D-Fauna · Animal reconstruction | 3D shape plausibility | Helped |
| Train category-agnostically instead of with category-specific conditioning (ablation) | 3D-Fauna · Rare animal categories | Reconstruction quality | Helped |
| Add the mask discriminator (ablation) | 3D-Fauna · Animal reconstruction | Viewpoint prediction and shape quality | Helped |
Surprising findings
- MagicPony, trained category-specifically, produces severe artifacts on rare categories with fewer than 100 images such as the puma.
- Conditioning on an explicit category embedding instead of the semantic bank gives sub-optimal reconstructions, especially for rare categories.
Limitations
- Limited to quadruped species that share the same skeletal structure.
- Training images still need light curation to remove blur and occlusion.
- Comparisons with some baselines are not numerically fair given differing settings.
Experimental setup
Datasets: Animals-with-Attributes APT-36K Animal3D DOVE Fauna Dataset PASCAL VOC
Benchmarks: PASCAL VOC APT-36K Animal3D
Metrics: KT-PCK@0.1 PCK@0.1
Baselines: MagicPony UMR A-CSM LASSIE Hi-LASSIE Zero-1-to-3
Builds on: MagicPony SMPL DINO CMR
Closest papers by meaning
- SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild (NeurIPS 2026 · similarity 0.89)
- Reconstructing Animals and the Wild (CVPR 2025 · similarity 0.88)
- Web-Scale Collection of Video Data for 4D Animal Reconstruction (NeurIPS 2025 · similarity 0.86)
- VAREN: Very Accurate and Realistic Equine Network (CVPR 2024 · similarity 0.85)
- Implicit Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes (NeurIPS 2024 · similarity 0.85)
- BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.