Research map / Video / Video generation
Fast 4D Mesh Generation by Spatio-Temporal Attention Chains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free method that reads correspondences from the attention maps of a frozen 4D mesh generator (ActionMesh/TripoSG) via a V→Z→Z→V attention chain, animating an anchor mesh in ~9 s and enabling 2D/4D tracking and camera recovery.
Problem
Video-to-4D mesh generation is slow, expensive, drifts on long sequences, and outputs meshes with no link to input pixels or camera, requiring a separate topology-preserving animation stage.
Approach
Compose frozen attention maps (vertex-to-latent token, temporal latent-to-latent, latent-to-surface) into a Markov-style attention chain that yields dense anchor-to-frame correspondences, replacing the learned Stage II with closed-form landmark tracking and geodesic-rigid skinning, plus attention reinforcement for long rollouts.
Key results
- Generates a 16-frame 4D mesh in ~9 s vs ~120 s for ActionMesh, with CD-3D 0.048 and CD-4D 0.077 on ActionBench (best among compared methods), though CD-M is slightly worse (0.163 vs 0.148).
- Zero-shot DAVIS-foreground 2D tracking reaches OA 90.41, best overall, while AJ 53.33 trails supervised CowTracker (60.08).
- On 4D tracking, zero-shot method improves over ActionMesh Stage II by +28.4 APD3D on PointOdyssey and +23.7 on Dynamic Replica, outperforming TraceAnything but trailing supervised V-DPM.
Surprising findings
- Temporal correspondences inside the 4D backbone are already usable after only about four denoising steps, before the generated meshes look visually accurate.
- Dropping the learned Stage II entirely leads to slightly worse CD-Motion than ActionMesh, which the authors attribute to lack of learned temporal regularization.
Limitations
- Mesh quality depends on the frozen image-to-3D and denoiser models.
- Sparse smoothing and local-rigid deformation can damp fine motion; motion smoothness trails ActionMesh on CD-Motion.
- Long rollouts may still degrade as attention over generated anchors becomes diffuse.
Experimental setup
Datasets: ActionBench Consistent4D BADJA TAP-Vid-DAVIS PointOdyssey Dynamic Replica
Benchmarks: ActionBench Consistent4D BADJA TAP-Vid-DAVIS PointOdyssey Dynamic Replica
Metrics: CD-3D CD-4D CD-M Normal Consistency LPIPS CLIP DreamSim AJ delta avg OA
Baselines: ActionMesh ShapeGen4D L4GM LIM DreamMesh4D V2M4 Step1X-3D GVFD TRELLIS TripoSG Denoise-to-Track DiffTrack
Builds on: ActionMesh TripoSG As-Rigid-As-Possible DiffTrack
Closest papers by meaning
- Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generation (NeurIPS 2025 · similarity 0.88)
- 4Diffusion: Multi-view Video Diffusion Model for 4D Generation (NeurIPS 2024 · similarity 0.87)
- 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.87)
- SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency (ICLR 2025 · similarity 0.87)
- DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.