Research map / 3D vision / Depth and stereo
Steerers: A Framework for Rotation Equivariant Keypoint Descriptors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a linear "steerer" in keypoint description space that approximates image rotations, enabling rotation-equivariant descriptors with a single descriptor pass, and sets new SOTA on AIMS and Roto-360.
Problem
Learned keypoint descriptors are not robust to in-plane camera rotation; augmentation hurts upright performance and test-time augmentation over rotations is expensive.
Approach
Treat rotation as a linear map on descriptions (a steerer), characterized via group representation theory for C4 and SO(2), and learn it either for a fixed descriptor, jointly with the descriptor, or fix the steerer and train the descriptor to match it.
Key results
- On AIMS, SO2-Freq1-B with Procrustes matcher reaches 60 AP overall versus 52 for SE2-LoFTR, with 59 on All Others versus 51.
- On Roto-360 with DeDoDe-SO2 detector, SO2-Spread-B reaches 95/97/98 at 3/5/10 px, outperforming RELF multiple (92/93/94 with SuperPoint).
- On MegaDepth-SO2, SO2-Spread-B with max similarity C8 steering gets 43/61/74 AUC, while the original DeDoDe-B with dual softmax gets 12/16/20 AUC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train DeDoDe-B with a C4 steerer and rotation augmentation (ablation) | DeDoDe-B · MegaDepth-1500 | AUC @ 5°: 49 -> 51 | Helped |
| Train DeDoDe-B with a C4 steerer and rotation augmentation (ablation) | DeDoDe-B · MegaDepth-C4 | AUC @ 5°: 12 -> 50 | Helped |
| Train DeDoDe-B with an SO2 steerer and continuous rotation augmentation (ablation) | DeDoDe-B · MegaDepth-SO2 | AUC @ 5°: 12 -> 44 | Helped |
| Use an invariant steerer instead of a spread-frequency SO2 steerer (ablation) | SO2-Spread-B · MegaDepth-SO2 | AUC @ 5°: 46 -> 43 | Hurt |
| Use a frequency-1 SO2 steerer instead of a spread-frequency SO2 steerer (ablation) | SO2-Spread-B · MegaDepth-SO2 | AUC @ 5°: 46 -> 45 | Hurt |
| Use a fixed C4 Perm steerer instead of a jointly optimized steerer (ablation) | C4-B · MegaDepth-1500 | AUC @ 5°: 51 -> 50 | No clear effect |
| Use a fixed C4 Perm steerer instead of a jointly optimized steerer (ablation) | C4-B · MegaDepth-C4 | AUC @ 5°: 50 -> 50 | No clear effect |
| Use an SO2 Spread steerer instead of an SO2 steerer initialized and jointly optimized (ablation) | SO2-B · MegaDepth-SO2 | AUC @ 5°: 44 -> 46 | Helped |
| Use max similarity rather than max matches for C8-steered matching (ablation) | SO2-Spread-B · MegaDepth-C4 | AUC @ 5°: 49 -> 47 | Hurt |
| Use Procrustes matching rather than max matches with C8-steered descriptions (ablation) | SO2-Freq1-B · MegaDepth-SO2 | AUC @ 5°: 45 -> 45 | No clear effect |
| Use a fixed invariant C4 steerer rather than a Perm steerer (ablation) | C4-Perm-B · MegaDepth-C4 | AUC @ 5°: 50 -> 47 | Hurt |
| Train DeDoDe-G with a fixed C4 Perm steerer (ablation) | DeDoDe-G · MegaDepth-C4 | AUC @ 5°: 13 -> 53 | Helped |
Surprising findings
- Steerers can be found post hoc for the original non-rotation-augmented DeDoDe descriptors, yielding approximate rotation equivariance without retraining.
- Joint optimization of steerer and descriptor has many local optima and strong initialization sensitivity in eigenvalue structure.
Limitations
- Joint optimization of steerer is sensitive to initialization and yields local optima with non-ideal eigenvalue structure.
- Invariant (identity steerer) models perform worse on upright images than other steerers.
- Per-correspondence relative rotations from the Procrustes matcher are not exploited further.
Experimental setup
Datasets: MegaDepth HPatches AIMS Roto-360
Benchmarks: AIMS Roto-360 MegaDepth-1500 MegaDepth-C4 MegaDepth-SO2
Metrics: average precision percentage of correct matches AUC @ 5/10/20 degrees
Baselines: SIFT ORB SuperPoint + RELF SE2-LoFTR DeDoDe-B DeDoDe-G
Builds on: DeDoDe SE2-LoFTR RELF Upright SIFT
Closest papers by meaning
- Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform (CVPR 2024 · similarity 0.84)
- DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers (NeurIPS 2026 · similarity 0.84)
- Selective Disk Bispectrum: A Complete and Rotation Invariant Image Descriptor (NeurIPS 2026 · similarity 0.84)
- Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis (NeurIPS 2026 · similarity 0.84)
- Estimating Extreme 3D Image Rotations using Cascaded Attention (CVPR 2024 · similarity 0.83)
- Cameras as Relative Positional Encoding (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.