Research map / 3D vision / Face and shape
LIM: Large Interpolator Model for Dynamic Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LIM, a transformer feed-forward interpolator of triplane 3D representations between keyframes, trained with a causal consistency loss, enabling continuous-time 4D reconstruction and canonical-coordinate mesh tracking.
Problem
Feed-forward reconstruction of time-continuous, tracked 4D assets from video; existing 4D reconstructors are category-specific or optimization-based and cannot interpolate or produce time-consistent meshes.
Approach
A LIM module built on a pretrained multi-view LRM that cross-attends LRM intermediate features at keyframe k to target keyframe images to predict an interpolated triplane at any continuous time, trained with a self-supervised causal consistency loss that feeds its own intermediate features back in; a second canonical-coordinate LRM/LIM pair enables mesh tracking.
Key results
- LIM reaches PSNR 23.11 and LPIPS 0.075 on triplane interpolation, versus 22.05 PSNR / 0.082 LPIPS for FiLM and 20.96 / 0.093 for linear interpolation.
- Removing the causal consistency loss drops PSNR from 23.11 to 22.2.
- In monocular 4D reconstruction, LIM achieves LPIPS 0.142 and FVD 811.1, against 0.429 / 1136.3 for Consistent4D (about 1.5 hours) and 0.504 / 1427.2 for TripoSR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LIM instead of linear triplane interpolation | Multi-view LRM · Triplane interpolation | PSNR: 20.96 -> 23.11 | Helped |
| Use LIM instead of linear triplane interpolation | Multi-view LRM · Triplane interpolation | LPIPS: 0.093 -> 0.075 | Helped |
| Use LIM mesh tracing instead of nearest-neighbor tracing | Deformable mesh tracking · Deformable mesh tracking | PSNR: 20.33 -> 21.56 | Helped |
| Use LIM mesh tracing instead of nearest-neighbor tracing | Deformable mesh tracking · Deformable mesh tracking | LPIPS: 0.122 -> 0.096 | Helped |
| Use LIM with multi-view diffusion and LRM instead of Consistent4D | Monocular video reconstruction · Monocular video reconstruction | LPIPS: 0.429 -> 0.142 | Helped |
| Use LIM with multi-view diffusion and LRM instead of Consistent4D | Monocular video reconstruction · Monocular video reconstruction | FVD: 1136.3 -> 811.1 | Helped |
| Add causal consistency loss Lcausal (ablation) | LIM · Triplane interpolation | PSNR: 22.2 -> 23.11 | Helped |
| Add causal consistency loss Lcausal (ablation) | LIM · Triplane interpolation | PSNRFG: 15.38 -> 16.12 | Helped |
| Add causal consistency loss Lcausal (ablation) | LIM · Triplane interpolation | LPIPS: 0.084 -> 0.075 | Helped |
Surprising findings
- Linear interpolation in triplane space often makes dynamic parts disappear, and image-space interpolation (FiLM) produces ghosting artifacts.
Limitations
- Trained only on synthetic data; it has not been tested on real-world videos since no real-data LRM is available.
- Performs interpolation only, not extrapolation.
- Mesh tracking degrades around thin structures.
Experimental setup
Datasets: Objaverse-like artist-created animated meshes (synthetic)
Benchmarks: held-out animated object set (256 scenes) 8-step mesh tracking test sequences monocular video reconstruction set (16-frame videos)
Metrics: PSNR PSNR_FG LPIPS FVD
Baselines: FiLM Linear triplane interpolation Consistent4D TripoSR Nearest-neighbor tracing
Builds on: Large Reconstruction Model (LRM) FiLM TripoSR Lightplane renderer
Closest papers by meaning
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.87)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.87)
- 4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos (NeurIPS 2025 · similarity 0.86)
- 4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time (NeurIPS 2025 · similarity 0.86)
- L4GM: Large 4D Gaussian Reconstruction Model (NeurIPS 2024 · similarity 0.86)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.