Research map / 3D vision / Face and shape
As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes APAP, a handle-based mesh deformation method that optimizes per-face Jacobians with SDS gradients from a LoRA-finetuned Stable Diffusion prior plus a handle-displacement loss, through a differentiable Poisson solve.
Problem
Classical geometry-based mesh deformation (e.g., ARAP) ignores perceptual plausibility, producing unnatural distortions under user handle drags, while learning-based deformation requires category-specific shape collections.
Approach
Treat per-face Jacobians as learnable parameters, recover vertices via a differentiable Poisson solve, and use SDS from a pretrained 2D diffusion model (LoRA-finetuned on renders of the source mesh for identity) as a plausibility critic, jointly with a handle-target soft constraint in a two-stage optimization.
Key results
- On APAP-BENCH 2D, APAP achieves k-NN GIQA 4.887 (x10^-2) vs 4.753 for ARAP and 4.545 for DragDiffusion.
- In a MTurk user study with 102 valid responses, APAP was preferred 59.17% vs 40.83% for ARAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ARAP deformation with APAP | 2D mesh deformation · APAP-BENCH 2D | k-NN GIQA (×10−2): 4.753 -> 4.887 | Helped |
| Replace DragDiffusion with APAP | 2D mesh deformation · APAP-BENCH 2D | k-NN GIQA (×10−2): 4.545 -> 4.887 | Helped |
| Replace ARAP deformation with APAP | 2D image editing · APAP-BENCH 2D | preference: 40.83 -> 59.17 | Helped |
| Add the diffusion prior to Lh-only optimization (ablation) | APAP · APAP-BENCH 2D | k-NN GIQA (×10−2): 4.797 -> 4.887 | Helped |
| Replace Lh initialization with an ARAP initialization (ablation) | APAP · APAP-BENCH 2D | k-NN GIQA (×10−2): 4.887 -> 4.740 | Hurt |
| Replace Lh initialization with a constrained Poisson initialization (ablation) | APAP · APAP-BENCH 2D | k-NN GIQA (×10−2): 4.887 -> 4.316 | Hurt |
Surprising findings
- Initializing with ARAP or a constrained Poisson solve instead of the handle-loss first stage degrades plausibility (k-NN GIQA 4.740 and 4.316), since the diffusion prior struggles with distorted renderings.
Limitations
- Evaluation is limited to a small 2D benchmark of 80 meshes and a qualitative/small-scale 3D comparison against only ARAP.
- Relies on SDS gradients from a single 2D diffusion model, which can struggle with distorted renderings.
- Uses an explicit per-face Jacobian optimization that depends on user-specified handles and anchors.
Experimental setup
Datasets: ShapeNet Genie APAP-BENCH 2D (generated with SDXL)
Benchmarks: APAP-BENCH 3D APAP-BENCH 2D
Metrics: k-NN GIQA score (k=12) user preference (%)
Baselines: ARAP DragDiffusion
Builds on: score distillation sampling as-rigid-as-possible dreambooth lora latent diffusion models
Closest papers by meaning
- DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · similarity 0.85)
- Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates (NeurIPS 2025 · similarity 0.85)
- DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction (NeurIPS 2024 · similarity 0.85)
- DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion (CVPR 2025 · similarity 0.84)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.84)
- Diffeomorphic Mesh Deformation via Efficient Optimal Transport for Cortical Surface Reconstruction (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.