Research map / Label-efficient and robust learning / Domain adaptation and generalization
SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSHNet, which reformulates unsupervised cross-modal homography estimation into two supervised sub-problems (homography estimation and modality transfer) trained via split optimization, plus homography feature supervision and distillation.
Problem
Unsupervised cross-modal homography estimation lacks ground-truth homographies, and intensity/content-consistency losses have many local minima under large modality and deformation gaps.
Approach
Reformulate the unsupervised problem into a mono-modal homography estimation sub-problem and a modality transfer sub-problem, each with direct supervision, and train the two networks alternately with split optimization (one frozen while the other updates), using a two-branch self-supervised regularizer to bootstrap convergence; add correlation-based homography feature loss and distill into a stude
Key results
- SSHNet-IHN achieves MACE 2.94 on OPT-SAR, 47.4% lower than supervised MHN and 85.8% lower than supervised LocalTrans.
- SSHNet-IHN reaches 1.23 MACE on GoogleMap and 1.66 on RGB/NIR, outperforming all listed unsupervised methods by a large margin.
- Distilled SSHNet-IHN-D trained on DPDN improves ACE<5 reliable-estimation ratio by 66.5% on GoogleMap and 92.1% on Flash/no-flash relative to SSHNet-IHN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SSHNet-IHN instead of supervised MHN | Homography estimation · OPT-SAR | MACE: 5.59 -> 2.94 | Helped |
| Use SSHNet-IHN instead of supervised LocalTrans | Homography estimation · OPT-SAR | MACE: 20.68 -> 2.94 | Helped |
| Add the SSHNet training framework to SCPNet | SCPNet homography architecture · GoogleMap, DPDN, OPT-SAR, Flash/no-flash, RGB/NIR | MACE: 4.35 -> 3.89; NC -> 4.81; NC -> 12.94; 2.67 -> 2.65; 4.78 -> 3.86 | Helped |
| Remove problem reformulation (ablation) | SSHNet · OPT-SAR | MACE: 2.94 -> NC | Hurt |
| Replace split optimization with simultaneous optimization (ablation) | SSHNet · OPT-SAR | MACE: 2.94 -> NC | Hurt |
| Use pure self-supervised learning instead of SSHNet training (ablation) | SSHNet · OPT-SAR | MACE: 2.94 -> 24.60 | Hurt |
| Replace Transformer-based transfer network with CNN-based network (ablation) | SSHNet modality transfer network · OPT-SAR | MACE: 2.94 -> 8.72 | Hurt |
| Use perceptual loss instead of L1 transfer loss (ablation) | SSHNet modality transfer network · OPT-SAR | MACE: 5.88 -> 4.52 | Helped |
| Use perceptual loss instead of L2 transfer loss (ablation) | SSHNet modality transfer network · OPT-SAR | MACE: 6.23 -> 4.52 | Helped |
| Add homography feature-space supervision (ablation) | SSHNet · OPT-SAR | MACE: 4.52 -> 2.94 | Helped |
| Use correlation-based HF loss instead of L1 HF loss (ablation) | SSHNet · OPT-SAR | MACE: 4.09 -> 2.94 | Helped |
| Use correlation-based HF loss instead of L2 HF loss (ablation) | SSHNet · OPT-SAR | MACE: 3.27 -> 2.94 | Helped |
Surprising findings
- Straight simultaneous optimization of both sub-problems fails to converge, and pure self-supervised training reaches MACE 24.60 on OPT-SAR.
- Transferring from the richer modality to the less informative one (e.g., RGB->NIR) gives better results than the reverse direction.
Limitations
- Cross-domain generalization remains limited between challenging datasets due to large domain gaps.
- Performance on real-world parallax scenes (UDIS-TIR) is evaluated only with a single model, and SCPNet fails to converge there.
Experimental setup
Datasets: GoogleMap DPDN OPT-SAR Flash/no-flash RGB/NIR UDIS
Benchmarks: GoogleMap DPDN OPT-SAR Flash/no-flash RGB/NIR UDIS-TIR
Metrics: MACE PSNR SSIM ACE<5 ratio
Baselines: SIFT ORB UDHN CA-UDHN biHomE BasesHomo UMF-CMGR SCPNet DHN MHN LocalTrans IHN
Builds on: IHN RHWF SCPNet Swin-Unet perceptual loss
Closest papers by meaning
- MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation (CVPR 2024 · similarity 0.90)
- Unsupervised Homography Estimation on Multimodal Image Pair via Alternating Optimization (NeurIPS 2024 · similarity 0.89)
- Adapting Dense Matching for Homography Estimation with Grid-based Acceleration (CVPR 2025 · similarity 0.87)
- Mask-Homo: Pseudo Plane Mask-Guided Unsupervised Multi-Homography Estimation (AAAI 2024 · similarity 0.84)
- Noisy One-point Homographies are Surprisingly Good (CVPR 2024 · similarity 0.84)
- HUNet: Homotopy Unfolding Network for Image Compressive Sensing (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.