Research map / Image generation / Autoregressive and latent generation
FastJAM: a Fast Joint Alignment Model for Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FastJAM, a GNN-based joint image alignment method that builds a sparse keypoint graph from RoMa matches and DP-Means clustering, trained with a sparse inverse-compositional Geman-McClure loss to align image collections in seconds.
Problem
Joint alignment of image collections into a shared coordinate frame is slow (hours to minutes) and relies on heavy regularization and hyperparameter tuning, with dense features being memory-intensive.
Approach
Build a graph over sparse keypoints (intra-image edges from clustering, inter-image edges from pairwise matches) and use a GraphSAGE GNN with per-image mean pooling to predict homographies, optimized with a sparse keypoint inverse-compositional loss that needs no regularization term.
Key results
- FastJAM achieves average PCK@0.10 of 53.0 on SPair-71k test set, the best among test-time optimization methods (SpaceJAM ViT-L 45.7, ASIC 37.0).
- FastJAM aligns three SPair-71k categories in about 49 seconds versus about 6 minutes for SpaceJAM and over an hour for ASIC and Neural Congealing.
- On CUB-200 (first 3 categories), FastJAM reaches 75.3 PCK, comparable to ASIC's 75.9 and above SpaceJAM's 69.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FastJAM instead of ASIC | joint image alignment · SPair-71k test set (All) | PCK@0.10: 37.0 -> 53.0 | Helped |
| use FastJAM instead of SpaceJAM (ViT-L) | joint image alignment · SPair-71k test set (All) | PCK@0.10: 45.7 -> 53.0 | Helped |
| use FastJAM instead of SpaceJAM | joint image alignment · CUB-200 (first 3 cate.) | PCK: 69.6 -> 75.3 | Helped |
| replace RoMa with LoFTR (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 33.8; 53.0 -> 17.0 | Hurt |
| use linear projection (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 33.2; 53.0 -> 14.0 | Hurt |
| replace graph GNN with per-image MLP (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 73.5; 53.0 -> 47.8 | Hurt |
| remove deep network and optimize homographies directly (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 74.1; 53.0 -> 48.3 | Hurt |
| replace Geman-McClure loss with L2 loss (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 61.0; 53.0 -> 33.9 | Hurt |
| remove object masks (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 74.8; 53.0 -> 41.8 | Hurt |
| remove non-maximum suppression (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 74.3; 53.0 -> 47.2 | Mixed |
| remove Lie-algebraic parameterization (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 75.0; 53.0 -> 49.3 | Hurt |
| remove intra-image edges (ablation) | FastJAM · CUB-200 and SPair-71k (first 3 cate.) | PCK: 75.3 -> 74.7; 53.0 -> 50.0 | Mixed |
Surprising findings
- Removing the Lie-algebraic parameterization barely changes accuracy but about 2% of runs fail due to non-invertible matrices.
- LoFTR as matcher drops performance sharply (17.0 on SPair-71k subset) compared to RoMa, showing matcher quality dominates.
Limitations
- Depends on the quality of the external image matcher (RoMa) for initial correspondences.
- Homography transformations may be insufficient for strong non-planar deformations.
- Struggles with highly symmetric objects such as bicycles due to matcher ambiguity.
Experimental setup
Datasets: SPair-71k CUB-200
Benchmarks: SPair-71k CUB-200
Metrics: PCK@0.10 runtime
Baselines: Neural Congealing ASIC SpaceJAM GANgealing Neural Best Buddies DINO+NN LoFTR
Builds on: SpaceJAM RoMa GraphSAGE DP-Means Least-Squares Congealing
Closest papers by meaning
- FC-GNN: Recovering Reliable and Accurate Correspondences from Interferences (CVPR 2024 · similarity 0.85)
- Adapting Dense Matching for Homography Estimation with Grid-based Acceleration (CVPR 2025 · similarity 0.85)
- GIM: Learning Generalizable Image Matcher From Internet Videos (ICLR 2024 · similarity 0.84)
- Towards Optimizing Large-Scale Multi-Graph Matching in Bioimaging (CVPR 2025 · similarity 0.84)
- DH-Set: Improving Vision-Language Alignment with Diverse and Hybrid Set-Embeddings Learning (CVPR 2025 · similarity 0.84)
- Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.