Research map / Video / Video generation
EG4D: Explicit Generation of 4D Object without Score Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates a 4D object from one image by explicitly synthesizing temporally consistent multi-view videos with SVD and SV3D (attention injection, training-free), reconstructing with 4D Gaussian Splatting plus color transformation, and refining with SDXL-Turbo, avoiding score distillation.
Problem
Image-to-4D generation with score distillation suffers from over-saturation and the Janus problem, and video-diffusion-generated multi-view frames are temporally inconsistent and semantically defective.
Approach
Replace SDS with explicit generation: SVD produces dynamics, SV3D produces multi-views per frame with training-free spatial KV latent EMA injection for temporal consistency; a 4DGS reconstruction adds a time-specific color affine transform and multiscale augmentation, followed by one-pass diffusion refinement.
Key results
- CLIP-I of 0.9535 for EG4D vs 0.9227 for DreamGaussian4D, 0.9214 for Consistent4D, 0.8544 for Animate124.
- User study: EG4D preferred in 83.75% of overall quality votes vs 11.27% for DreamGaussian4D.
- Full pipeline takes about 1 hour 25 minutes per object on a single RTX 3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use spatial KV latent blending with EMA at α=0.5 instead of no attention injection (ablation) | EG4D video generation · Generated multi-view videos | temporal consistency | Helped |
| Use temporal-attention EMA blending instead of spatial-attention blending (ablation) | EG4D video generation · Generated multi-view videos | motion range | Hurt |
| Blend the residual connection instead of KV latent (ablation) | EG4D video generation · Generated multi-view videos | generation quality | Hurt |
| Use first-frame-only linear KV blending instead of EMA blending (ablation) | EG4D video generation · Generated multi-view videos | generation capability for large motions | Hurt |
| Increase the EMA blending weight above 0.5 (ablation) | EG4D video generation · Generated multi-view videos | motion range | Hurt |
| Add time-specific affine color transformation (ablation) | EG4D 4D reconstruction · Generated dynamic 4D assets | temporal texture consistency | Helped |
| Add multiscale rendering augmentation (ablation) | EG4D 4D reconstruction · Video-diffusion reconstruction | test PSNR | Helped |
| Use adversarial training for refinement instead of one-pass diffusion refinement (ablation) | EG4D refinement · tiger-guitar | rendered image quality | Hurt |
| Use SDS refinement instead of one-pass diffusion refinement (ablation) | EG4D refinement · tiger-guitar | semantic detail refinement | Hurt |
| Use iterative refinement instead of one-pass refinement (ablation) | EG4D refinement · tiger-guitar | rendering sharpness | Hurt |
| Use textured mesh extraction instead of one-pass diffusion refinement (ablation) | EG4D refinement · tiger-guitar | appearance coherence | Hurt |
| Add one-pass diffusion refinement (ablation) | EG4D refinement · tiger-guitar | semantic detail quality | Helped |
Surprising findings
- Setting attention-injection weight above 0.5 significantly reduces motion range, a consistency-motion trade-off.
- GAN discriminator refinement caused Gaussians to diverge from the surface and renders to turn black.
Limitations
- Cannot generate high-dynamic motion due to base image-to-video model and consistency-motion trade-off.
- Multi-view diffusion struggles with precise camera pose conditioning, leading to unsatisfactory reconstruction.
- Stage I video and multi-view generation is a runtime bottleneck.
Experimental setup
Datasets: input images from Animate124 and DreamGaussian4D demos SVD-generated videos
Benchmarks: user preference study (47 participants) qualitative cases zelda, monkey-bike, tiger-guitar
Metrics: CLIP-I user preference percentage
Baselines: Animate124 Consistent4D DreamGaussian4D
Builds on: 4D Gaussian Splatting Stable Video Diffusion SV3D HexPlane 3D Gaussian Splatting
Closest papers by meaning
- SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency (ICLR 2025 · similarity 0.92)
- Diffusion$^2$: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models (ICLR 2025 · similarity 0.90)
- Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.90)
- 4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling (CVPR 2024 · similarity 0.90)
- 4Diffusion: Multi-view Video Diffusion Model for 4D Generation (NeurIPS 2024 · similarity 0.90)
- CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.