Research map / 3D vision / Face and shape
PolyPose: Deformable 2D/3D Registration via Polyrigid Transformations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PolyPose, an optimization-based deformable 2D/3D registration method that parameterizes the deformation as a log-Euclidean blend of per-bone rigid transforms, optimized via differentiable X-ray rendering without deformation regularizers.
Problem
Deformable 2D/3D registration of a preoperative 3D CT to sparse-view or limited-angle intraoperative X-rays is severely under-determined, and existing methods need extensive regularization and per-case hyperparameter tuning.
Approach
Model the 3D deformation as a weighted combination in se(3) of rigid transforms of individual bones (polyrigid warp), with a hyperparameter-free mass-based weight field w_k = m_k / (1 + d_k^2), so that bones stay rigid and the warp is smooth and invertible without regularizers.
Key results
- On DeepFluoro with two X-rays spanning ~30 degrees, PolyPose achieves pelvis Dice 0.99, left femur Dice 0.98 and right femur Dice 0.98 with 0.00% folds, outperforming baselines such as anatomix (femur Dice 0.93/0.92).
- Dense R^3 and Dense SE(3) parameterizations break the femurs and Dense SE(3) yields 44.08% folds on DeepFluoro, showing the low-dimensional polyrigid model is needed under minimal supervision.
- PolyPose is robust to segmentation erosion of 1-3 mm, outperforming the baselines in Dice and HD95 over that range.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a low-dimensional polyrigid deformation instead of dense R^3 translations (ablation) | 2D/3D registration on DeepFluoro · DeepFluoro, Femur (L) | Dice: 0.97(0.02) -> 0.98(0.01) | Helped |
| use a low-dimensional polyrigid deformation instead of dense SE(3) transformations (ablation) | 2D/3D registration on DeepFluoro · DeepFluoro, Femur (L) | Dice: 0.90(0.02) -> 0.98(0.01) | Helped |
| use mass-based weights instead of reciprocal-distance weights (ablation) | PolyPose · DeepFluoro, Femur (L) | Dice: 0.95(0.01) -> 0.98(0.01) | Helped |
| use mass-based weights instead of reciprocal-distance weights (ablation) | PolyPose · DeepFluoro, Femur (R) | Dice: 0.96(0.01) -> 0.98(0.01) | Helped |
| optimize rigid poses of additional anatomical structures beyond the skull, cervical spine, and thoracic spine (ablation) | PolyPose · Head&Neck | 3D Dice | No clear effect |
| include the skull, cervical spine, and thoracic spine as optimized rigid structures (ablation) | PolyPose · Head&Neck | deformation-field stability | Helped |
| erode CT structure segmentations by radii from 1 to 3 mm (ablation) | PolyPose · DeepFluoro | Dice | No clear effect |
| use PolyPose rather than Dense R^3 | 2D/3D registration · DeepFluoro, Femur (L) | Dice: 0.97(0.02) -> 0.98(0.01) | Helped |
| use PolyPose rather than Dense R^3 | 2D/3D registration · DeepFluoro, Femur (R) | Dice: 0.96(0.02) -> 0.98(0.01) | Helped |
Surprising findings
- Weight field based on mass-scaled inverse-square distance avoids per-structure hyperparameter tuning and outperforms the reciprocal-distance weighting with tuned epsilon.
- Dense deformation models break the femurs, and dense SE(3) produces 44% folds, while the polyrigid model stays diffeomorphic.
Limitations
- Cannot model motions that create empty space or split a rigid body, such as opening the jaw, since the warps are diffeomorphisms.
- May fail to constrain soft tissue far from skeletal structures (e.g., abdomen), with extreme deformations not fully evaluated.
- Requires segmentations of rigid bodies in the CT to build the weight field.
Experimental setup
Datasets: Head&Neck (31 patients, longitudinal CT) DeepFluoro (6 cadaveric subjects) ThoraxCBCT (13 patients)
Benchmarks: Head&Neck DeepFluoro ThoraxCBCT
Metrics: 3D Dice 2D Dice HD95 % Folds sigma(log|J|) runtime
Baselines: xvr LiftReg 2D3D-RegNet uniGradICON multiGradICON FireANTs anatomix Dense R^3 Dense SE(3)
Builds on: xvr polyrigid registration (3D/3D) Nerfies RAFT-3D LiftReg
Closest papers by meaning
- Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering (CVPR 2024 · similarity 0.90)
- An Unsupervised Learning Model for Deformable Medical Image Registration (CVPR 2018 · similarity 0.87)
- CARL: A Framework for Equivariant Image Registration (CVPR 2025 · similarity 0.85)
- LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators (NeurIPS 2026 · similarity 0.84)
- Uncertainty Estimation for Pretrained Medical Image Registration Models via Transformation Equivariance (NeurIPS 2026 · similarity 0.83)
- Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.