Research map / Architectures and efficiency / Attention and transformers
H-ViT: A Hierarchical Vision Transformer for Deformable Image Registration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H-ViT, a hybrid CNN-Transformer for unsupervised 3D deformable MRI registration that uses self-attention plus hierarchical cross-attention over a pyramid of deformation-field features.
Problem
Accurate representation of the deformation field in deformable medical image registration, where CNNs have limited receptive fields and standard self-attention Transformers capture long-range flow patterns poorly.
Approach
A top-down dual-attention block in which self-attention captures short-range flow within a scale and hierarchical cross-attention lets queries at each scale attend to keys/values from higher-level (coarser) deformation feature maps, propagating flow information across scales irrespective of spatial proximity.
Key results
- On OASIS, H-ViT reaches Dice 0.876 vs 0.862 for TransMorph (self-attention only).
- On IXI inter-patient registration, H-ViT achieves Dice 0.810 vs 0.793 for TransMorph-Bspl, with about 0.5% folded voxels.
- On LPBA inter-patient registration, H-ViT achieves Dice 0.704 vs 0.670 for TransMorph-Bspl; ablation shows cross-attention increases Dice from 0.801 (self-attention only) to 0.810 with three cross-attention units.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use H-ViT instead of TransMorph | deformable image registration · OASIS inter-patient | Dice: 0.862±0.014 -> 0.876±0.014 | Helped |
| Use H-ViT instead of TransMorph-Bspl | deformable image registration · IXI patient-to-atlas | Dice: 0.778±0.080 -> 0.797±0.075 | Helped |
| Add self-attention to the CNN backbone (ablation) | H-ViT · IXI registration | Dice: 0.785±0.081 -> 0.801±0.072 | Helped |
| Add cross-attention to self-attention (ablation) | H-ViT · IXI registration | Dice: 0.801±0.072 -> 0.810±0.073 | Helped |
| Increase cross-attention units from 1 to 2 (ablation) | H-ViT · IXI registration | Dice: 0.803±0.072 -> 0.806±0.073 | Helped |
| Increase cross-attention units from 2 to 3 (ablation) | H-ViT · IXI registration | Dice: 0.806±0.073 -> 0.810±0.073 | Helped |
| Increase number of heads from 8 to 64 (ablation) | small H-ViT · IXI registration | Dice: 0.801±0.072 -> 0.805±0.072 | Helped |
| Set depth to 4 instead of 1 (ablation) | small H-ViT · IXI registration | Dice: 0.803±0.073 -> 0.805±0.071 | Helped |
| Use voxel patches of size 6 × 6 × 6 instead of 2 × 2 × 2 (ablation) | small H-ViT · IXI registration | Dice: 0.803±0.073 -> 0.804±0.073 | No clear effect |
| Set drop rate to 0.2 instead of 0 (ablation) | small H-ViT · IXI registration | Dice: 0.803±0.073 -> 0.801±0.073 | Hurt |
Surprising findings
- Cross-attention units alone (without self-attention) reach a Dice score (0.802) comparable to self-attention alone (0.801), and cross-attention adds only modest FLOPs overhead (2.2X vs 1.7X).
Limitations
- Higher Jacobian-determinant folding percentage than some baselines (e.g., 0.525% vs TransMorph-Bspl near zero on IXI).
- Evaluation limited to brain T1 MRI datasets with a fixed VoxelMorph-style backbone and identical training settings.
Experimental setup
Datasets: OASIS IXI ADNI LPBA Mindboggle
Benchmarks: OASIS IXI ADNI LPBA Mindboggle
Metrics: Dice HD95 SDlogJ percentage of non-positive Jacobian determinant FLOPs parameters
Baselines: VoxelMorph MIDIR CycleMorph ViT-V-Net TransMorph-Bspl TransMorph-Bayes CoTr nnFormer PVT LapIRN ConvexAdam DFI-NFF
Builds on: VoxelMorph TransMorph stationary velocity field diffeomorphic registration Transformer self-attention
Closest papers by meaning
- Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration (CVPR 2024 · similarity 0.86)
- A Plug-and-Play Image Registration Network (ICLR 2024 · similarity 0.85)
- SACB-Net: Spatial-awareness Convolutions for Medical Image Registration (CVPR 2025 · similarity 0.84)
- CARL: A Framework for Equivariant Image Registration (CVPR 2025 · similarity 0.84)
- Deep Learning in Medical Image Registration: Magic or Mirage? (NeurIPS 2024 · similarity 0.84)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.