Research map / Detection and segmentation / CNN features

Towards Complementary Keypoint Detection via Mixture of Detectors

NeurIPS 2026 · Detection and segmentation › CNN features · approach: Region-based and query-based detectors

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mixture of Detectors (MoD), which distills six heterogeneous keypoint detectors into one VGG-based multi-head model and trains a patch-wise, budget-conditioned router with matching-based rewards to select a detector per 8x8 patch.

Problem

Single keypoint detectors yield suboptimal keypoint distributions because their complementary strengths vary with scene structure and keypoint budget; prior fusion methods use heuristic concatenation and no supervision exists for detector selection.

Approach

Treat point-, line-, and edge-based detectors as experts; distill them into a shared encoder with multiple heads for single-pass prediction, then learn a router that selects detectors per patch using budget tokens and rewards derived from downstream matching inlier counts and precision.

Key results

• MoD with RoMa matcher reaches 68.5 AUC@5 on MegaDepth at 8192 keypoints, versus 66.4 for the best single teacher DeDoDe at 8192 (and 66.0 for DaD).
• On HPatches with RCM+ matcher, MoD achieves 53.3 AUC@3 at 8192 keypoints, above DeDoDe (52.5) and DaD (51.6).
• The distilled student head (MoD†) outperforms its teacher detector in most settings, e.g., SP on HPatches at 512 keypoints goes from 58.6 to 60.4.

Surprising findings

• Using downstream pose estimation error as an additional router reward gave no observable improvement, suggesting indirect and noisy supervision.
• The distilled encoder can outperform individual teachers, which the authors attribute to aggregating knowledge from multiple teachers.

Limitations

• Downstream pose error as a reward did not improve results, so reward design is limited to matching inlier-based signals.
• Texture-less regions such as floors or walls are sometimes classified as background, losing useful keypoints.
• Router supervision depends on an off-the-shelf matcher (RoMa v2) used during training.

Experimental setup

Datasets: MegaDepth ScanNet HPatches

Benchmarks: MegaDepth ScanNet HPatches

Metrics: AUC@5 pose error AUC@3 pixel homography end-point error

Baselines: SuperPoint ALIKED DeDoDe DaD DeepLSD MuGE Canny UAED SIFT DISK RCM+

Builds on: Mixture of Experts DaD DeDoDe SuperPoint ALIKED DKM

Closest papers by meaning

• Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task (CVPR 2020 · similarity 0.87)
• Dense Keypoints via Multiview Supervision (NeurIPS 2021 · similarity 0.87)
• Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features (CVPR 2024 · similarity 0.85)
• HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation (AAAI 2025 · similarity 0.84)
• RUBIK: A Structured Benchmark for Image Matching across Geometric Challenges (CVPR 2025 · similarity 0.84)
• ViKIENet: Towards Efficient 3D Object Detection with Virtual Key Instance Enhanced Network (CVPR 2025 · similarity 0.83)