Research map / Video / Video segmentation and tracking
Recovering Dynamic 3D Sketches from Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Liv3Stroke reconstructs dynamic 3D sketches from monocular videos by guiding a set of deformable cubic Bézier strokes with a perceptually-supervised 3D point cloud motion field.
Problem
Abstracting and representing 3D object motion from videos (including moving cameras) in a compact, explicit form, since dense dynamic scene reconstruction is under-constrained and sensitive to appearance changes.
Approach
Represent motion as a set of deformable 3D Bézier strokes whose positions and control points are moved over time, initialized from a point-cloud motion guidance learned with LPIPS and CLIP perceptual losses rather than pixel losses, enabling view-consistent sketch animation.
Key results
- Liv3Stroke achieves per-frame Chamfer 0.252 and motion velocity distance 4.16e-3, comparable to DG-Mesh (0.277, 4.10e-3) and better than 4DGS on velocity.
- On structural alignment (novel views), Liv3Stroke scores 0.693 vs. 0.784 for Suggestive Contours, which the authors note tends to produce outline contours rather than abstractions.
- Liv3Stroke has the smallest deviation in motion prompt similarity across novel and fixed views (0.656 ± 0.006 novel).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Liv3Stroke instead of CLIPasso | Dynamic 3D sketch synthesis · Dynamic 3D sketches, novel views | Structural alignment: 0.760 ± 0.107 -> 0.693 ± 0.096 | Hurt |
| use Liv3Stroke instead of Suggestive Contours | Dynamic 3D sketch synthesis · Dynamic 3D sketches, novel views | Motion prompt similarity: 0.661 ± 0.013 -> 0.656 ± 0.006 | No clear effect |
| use Liv3Stroke instead of 4DGS | 3D motion guidance · Synthetic dataset | Per-frame Chamfer: 0.205 ± 0.046 -> 0.252 ± 0.049 | Hurt |
| use Liv3Stroke instead of DG-Mesh | 3D motion guidance · Synthetic dataset | Motion velocity distance (×10−3): 4.10 ± 2.70 -> 4.16 ± 2.34 | Hurt |
| remove temporal continuity loss from motion guidance (ablation) | Liv3Stroke · 3D motion guidance | Motion velocity distance (×10−3): 4.16 ± 2.34 -> 6.81 ± 5.41 | Hurt |
| remove rigid regularization from motion guidance (ablation) | Liv3Stroke · 3D motion guidance | Motion velocity distance (×10−3): 4.16 ± 2.34 -> 5.45 ± 3.74 | Hurt |
| replace the rigid loss formulation with L2 loss (ablation) | Liv3Stroke · 3D motion guidance | Motion velocity distance (×10−3): 4.16 ± 2.34 -> 4.61 ± 2.94 | Hurt |
| use masks instead of RGB images (ablation) | Liv3Stroke motion guidance · 3D motion guidance | Per-frame Chamfer: 0.252 ± 0.049 -> 0.257 ± 0.049 | No clear effect |
| use grayscale frames instead of RGB images (ablation) | Liv3Stroke motion guidance · 3D motion guidance | Motion velocity distance (×10−3): 4.16 ± 2.34 -> 4.41 ± 2.46 | Hurt |
| remove motion guidance (ablation) | Liv3Stroke sketch reconstruction · Dynamic 3D sketches | Structural representation | Hurt |
| remove coarse guidance (ablation) | Liv3Stroke sketch reconstruction · Dynamic 3D sketches | Structural representation | Hurt |
| remove temporal continuity loss from sketch reconstruction (ablation) | Liv3Stroke sketch reconstruction · Dynamic 3D sketches | Motion stability | Hurt |
Surprising findings
- Suggestive Contours scores higher on structural alignment than Liv3Stroke despite requiring input meshes, which the authors attribute to its outline-like output.
Limitations
- Only view-independent strokes are used, so view-dependent contours (e.g., of rounded objects) cannot be rendered.
- Ablations and quantitative gains are limited; structural alignment is lower than some baselines.
- Real-world results are qualitative only.
Experimental setup
Datasets: D-NeRF real-world dynamic scene videos from prior works
Benchmarks: authors' synthetic moving-camera dataset real-world dynamic scenes
Metrics: MS-SSIM structural alignment CLIP motion prompt similarity per-frame Chamfer distance motion velocity distance
Baselines: CLIPasso Sketch Video Synthesis Suggestive Contours 4DGS Deformable 3DGS SC-GS DG-Mesh
Builds on: 3Doodle CLIPasso D-NeRF LPIPS CLIP
Closest papers by meaning
- Fast Encoder-Based 3D from Casual Videos via Point Track Processing (NeurIPS 2024 · similarity 0.86)
- Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos (CVPR 2025 · similarity 0.86)
- Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach (NeurIPS 2025 · similarity 0.86)
- Breathing Life Into Sketches Using Text-to-Video Priors (CVPR 2024 · similarity 0.85)
- Articulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Video (NeurIPS 2026 · similarity 0.85)
- Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.