Research map / Video / Video generation
PhysAnimator: Physics-Guided Generative Cartoon Animation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates anime-style animation from a single illustration by combining image-space deformable body simulation (energy strokes, rigging points) with a sketch-guided Stable Video Diffusion ControlNet and a cartoon interpolation model.
Problem
Generating physically plausible, stylized animation from static anime illustrations, where purely data-driven video diffusion produces distortions and pure 2D animation tools require simple inputs.
Approach
Simulate anime objects as 2D deformable meshes to produce optical flow, warp the extracted sketch with that flow, and render texture-agnostic sketch sequences into colored frames with a sketch-guided video diffusion model, optionally enriched by a cartoon interpolation model.
Key results
- Ours achieves VideoScore visual quality 2.89 and temporal consistency 2.86, the best among compared methods.
- Ours FID 90.4 vs. Cinemo 49.5 (Cinemo lower FID due to near-static videos).
- In user study, the method is preferred over Motion-I2V at 95% (VQ), 94% (TC), 97% (MP), 96% overall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use physics-guided animation and sketch-guided rendering instead of Cinemo | Anime image-to-video generation · Anime image-to-video test set | visual quality (VideoScore): 2.85 -> 2.89 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Cinemo | Anime image-to-video generation · Anime image-to-video test set | temporal consistency (VideoScore): 2.80 -> 2.86 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Cinemo | Anime image-to-video generation · Anime image-to-video test set | factual consistency (VideoScore): 2.58 -> 2.64 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Cinemo | Anime image-to-video generation · Anime image-to-video test set | FID: 49.5 -> 90.4 | Hurt |
| Use physics-guided animation and sketch-guided rendering instead of Drag Anything | Anime image-to-video generation · Anime image-to-video test set | visual quality (VideoScore): 2.77 -> 2.89 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Drag Anything | Anime image-to-video generation · Anime image-to-video test set | temporal consistency (VideoScore): 2.45 -> 2.86 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Drag Anything | Anime image-to-video generation · Anime image-to-video test set | dynamic degree (VideoScore): 2.97 -> 2.48 | Hurt |
| Use physics-guided animation and sketch-guided rendering instead of DynamiCrafter | Anime image-to-video generation · Anime image-to-video test set | temporal consistency (VideoScore): 2.68 -> 2.86 | Helped |
| Use physics-guided animation and sketch-guided rendering instead of Motion-I2V | Anime image-to-video generation · Anime image-to-video test set | factual consistency (VideoScore): 2.39 -> 2.64 | Helped |
| Add cartoon interpolation for complementary dynamics (ablation) | PhysAnimator · Anime animation results | motion expressiveness | Helped |
| Remove cartoon interpolation for complementary dynamics (ablation) | PhysAnimator · Anime animation results | motion expressiveness | Hurt |
| Use sketch-guided rendering instead of direct image warping with inpainting (ablation) | PhysAnimator · Anime animation results | visual fidelity | Helped |
Surprising findings
- Cinemo achieves the best FID largely because it generates nearly static videos resembling the reference image.
Limitations
- Evaluated on a small test set of 20 anime images with no established benchmark.
- Segmentation inaccuracies cause warping artifacts that rely on the diffusion model to correct.
- 2D representation cannot fully capture 3D effects.
Experimental setup
Datasets: Sakuga-42M
Benchmarks: custom 20-image anime test set
Metrics: FID VideoScore visual quality VideoScore temporal consistency VideoScore dynamic degree VideoScore factual consistency user preference (2AFC)
Baselines: CinemoDrag DragAnything DynamiCrafter Motion-I2V
Builds on: Stable Video Diffusion LVCD ToonCrafter Segment Anything Fixed Corotated model
Closest papers by meaning
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.87)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.86)
- FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations (CVPR 2025 · similarity 0.85)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.85)
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (CVPR 2024 · similarity 0.85)
- PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.