Research map / 3D vision / Gaussian splatting

BEAST3D: animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

NeurIPS 2026 · 3D vision › Gaussian splatting · approach: Animatable Gaussian human avatars · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Beast3D self-supervisedly pretrains a DINOv3-fed, VGGT-style transformer to predict 3D Gaussian splats from calibrated multi-view animal videos by reconstructing held-out views, then transfers the features to novel view synthesis, multi-view pose estimation, and neural encoding.

Problem

Extracting rich 3D representations from sparse, calibrated multi-view laboratory recordings of animals is hard: supervised pose estimation needs heavy annotation, and general-purpose 3D reconstruction models trained on generic scenes fail on lab imagery and sparse views.

Approach

Use ground-truth camera calibration (Plücker ray maps) instead of learned camera pose estimation, a frozen DINOv3 encoder, and a VGGT geometry transformer that predicts per-patch 3D Gaussians trained by held-out-view rendering with SAM3-distilled foreground masks, yielding features for behavior analysis.

Key results

• Across subjects, Beast3D improves PSNR and LPIPS over E-RayZer and Pose Splatter on every dataset in novel view synthesis, and best SSIM on all but Chickadee.
• In pose estimation with 100 labeled instances, Beast3D achieves the best performance on nearly all datasets, with VGGT best only on Rat7M, and consistently outperforms BEAST.
• In neural encoding, Beast3D Gaussian features outperform 3D keypoints on both Cheese3D and Chickadee and match BEAST CLS tokens.

Surprising findings

• Multi-view models (VGGT, E-RayZer) do not consistently beat single-view DINOv3 and BEAST baselines on pose estimation despite extensive tuning.
• E-RayZer fine-tuning yields no improvement and occasionally degrades performance relative to zero-shot.

Limitations

• Pretraining compute is substantial (~32 h on 8 A100 GPUs per dataset), limiting accessibility.
• Requires SAM3 masks produced offline with manual correction for failed propagations.
• Requires known camera calibration, so it does not apply to uncalibrated settings.

Experimental setup

Datasets: Cheese3D Rat7M Chickadee Human3.6M

Benchmarks: Cheese3D Rat7M Chickadee Human3.6M

Metrics: PSNR SSIM LPIPS pixel error bits per spike

Baselines: E-RayZer Pose Splatter VGGT DINOv3 BEAST

Closest papers by meaning

• Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance (NeurIPS 2025 · similarity 0.92)
• SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild (NeurIPS 2026 · similarity 0.89)
• Web-Scale Collection of Video Data for 4D Animal Reconstruction (NeurIPS 2025 · similarity 0.88)
• Reconstructing Animals and the Wild (CVPR 2025 · similarity 0.87)
• Implicit Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes (NeurIPS 2024 · similarity 0.86)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.86)