Research map / 3D vision / 3D generation
PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PuzzleFusion++, an auto-agglomerative 3D fracture assembly method that alternates a diffusion denoiser for 6-DoF fragment alignment with a transformer verifier that merges verified pairs, evaluated on Breaking Bad.
Problem
3D fracture assembly: recovering 6-DoF alignments of many fractured fragments from point clouds, where combinatorial search and local-geometry ambiguity make multi-fragment assembly hard.
Approach
Combines a diffusion model that jointly denoises 6-DoF alignments of all fragments with a transformer verifier that classifies pairwise alignments and merges confident pairs into larger groups, repeated iteratively (agglomerative plus auto-regressive), with anchor fragments fixing the coordinate frame.
Key results
- On Breaking Bad everyday subset, PuzzleFusion++ reaches PA 70.6% and CD 6.03e-3 vs. Jigsaw's 57.3% and 13.3e-3.
- Denoiser-only (1 iteration) achieves PA 67.3% vs. DiffAssemble's 27.5%.
- Ground-truth verifier upper bound reaches PA 82.9%, showing verifier quality is the main bottleneck.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full six-iteration auto-agglomerative process instead of one iteration (ablation) | PuzzleFusion++ · Breaking Bad everyday | PA (%): 67.3 -> 70.6 | Helped |
| Add the customized noise scheduler instead of no scheduler (ablation) | PuzzleFusion++ denoiser · Breaking Bad everyday | PA (%): 57.5 -> 67.3 | Helped |
| Add the pretrained autoencoder instead of removing it (ablation) | PuzzleFusion++ denoiser · Breaking Bad everyday | PA (%): 56.9 -> 67.3 | Helped |
| Add an anchor fragment instead of removing it (ablation) | PuzzleFusion++ denoiser · Breaking Bad everyday | PA (%): 61.1 -> 67.3 | Helped |
| Increase denoiser sampling steps from 5 to 20 (ablation) | PuzzleFusion++ denoiser-only inference · Breaking Bad everyday | PA (%): 62.4 -> 67.3 | Helped |
| Increase denoiser sampling steps from 20 to 50 (ablation) | PuzzleFusion++ denoiser-only inference · Breaking Bad everyday | PA (%): 67.3 -> 67.2 | No clear effect |
| Replace the Jigsaw verifier with a ground-truth verifier (ablation) | PuzzleFusion++ (6 iterations) · Breaking Bad everyday | PA (%): 70.6 -> 82.9 | Helped |
| Use the customized scheduler instead of a linear scheduler (ablation) | PuzzleFusion++ denoiser · Breaking Bad everyday | PA (%): 57.5 -> 67.3 | Helped |
| Use the customized scheduler instead of a cosine scheduler (ablation) | PuzzleFusion++ denoiser · Breaking Bad everyday | PA (%): 64.4 -> 67.3 | Helped |
| Use PuzzleFusion++ rather than Jigsaw | Breaking Bad everyday · Breaking Bad everyday | PA (%): 57.3 -> 70.6 | Helped |
| Use PuzzleFusion++ rather than Jigsaw | Breaking Bad everyday · Breaking Bad everyday | CD (×10^-3): 13.3 -> 6.03 | Helped |
| Use PuzzleFusion++ rather than Jigsaw | Breaking Bad artifact, trained on everyday · Breaking Bad artifact | CD (×10^-3): 14.3 -> 14.5 | Hurt |
Surprising findings
- Generalization to the artifact subset: Jigsaw marginally beats PuzzleFusion++ on CD (14.5 vs 14.3 in the table's ordering) and is clearly better on the other metrics.
- Rotation error remains high (RMSE(R) about 38 degrees) and Jigsaw beats it on rotation RMSE in the main comparison's rotation-focused analysis, despite large gains on other metrics.
Limitations
- Verifier accuracy is limited (recall 45.95% at threshold 0.9), missing many true matches and limiting gains from iterations.
- No backtracking, so early merge errors cannot be corrected.
- Rotation accuracy lags behind optimization-based Jigsaw; performance degrades with more fragments.
Experimental setup
Datasets: Breaking Bad
Benchmarks: Breaking Bad
Metrics: RMSE (rotation) RMSE (translation) part accuracy chamfer distance speed
Baselines: Global LSTM DGL SE(3)-Equiv DiffAssemble Jigsaw PMTR PHFormer FRASIER
Builds on: PuzzleFusion DiffAssemble Jigsaw PointNet++ VQ-VAE
Closest papers by meaning
- Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers (AAAI 2024 · similarity 0.87)
- Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving (NeurIPS 2024 · similarity 0.86)
- DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly (CVPR 2024 · similarity 0.85)
- PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts (ICLR 2026 · similarity 0.82)
- 3DPGS: 3D Probabilistic Graph Search for Archaeological Piece Grouping (AAAI 2025 · similarity 0.82)
- Puzzles: Unbounded Video-Depth Augmentation for Scalable End-to-End 3D Reconstruction (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.