Research map / Detection and segmentation / CNN features
Weakly Misalignment-free Adaptive Feature Alignment for UAVs-based Multimodal Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OAFA, which models cross-modal spatial offsets in a common subspace of RGB-IR features and uses them to guide deformable alignment and fusion, achieving adaptive alignment without strict alignment supervision for UAV multimodal object detection.
Problem
RGB-IR object detection from UAVs suffers from weak misalignment between image pairs and modality gap, which corrupts cross-modal offset estimation and degrades detection; existing methods rely on strict alignment that needs extra annotations.
Approach
Decouple RGB and IR features into modality-invariant (common subspace) and modality-specific parts, estimate basic spatial offsets only from the invariant features, and feed them as explicit initialization to an offset-guided deformable convolution that adaptively aligns RGB features for fusion, trained without any alignment loss.
Key results
- OAFA reaches 79.4% mAP on DroneVehicle, 3.3 points above the second-best method (SLBAF-Net-based fusion at 76.1%, with TSFADet and others lower).
- Baseline SLBAF-Net improves from 76.1% to 79.4% mAP with CSOM plus ODAF and two-stage training; CSOM alone gives 78.2% and ODAF alone 77.8%.
- Under simulated RGB position shifts up to ±15 pixels, OAFA's performance fluctuates little while the baseline declines noticeably; OAFA runs at 33.1 FPS on an RTX A6000.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SOM (ablation) | Baseline · DroneVehicle validation | mAP (%): 76.1 -> 76.5 | Helped |
| add DML (ablation) | Baseline · DroneVehicle validation | mAP (%): 76.1 -> 77.7 | Helped |
| add CSOM, combining DML and SOM (ablation) | Baseline · DroneVehicle validation | mAP (%): 76.1 -> 78.2 | Helped |
| add ODAF (ablation) | Baseline · DroneVehicle validation | mAP (%): +1.7% | Helped |
| use CSOM and ODAF without two-stage training (ablation) | Baseline · DroneVehicle validation | mAP (%): 79.4 -> 77.3 | Hurt |
| use two-stage training with CSOM and ODAF (ablation) | Baseline · DroneVehicle validation | mAP (%): 77.3 -> 79.4 | Helped |
Surprising findings
- Naive two-stage-free end-to-end training drops accuracy to 77.3%, below the CSOM-only variant, indicating training instability of the deformable alignment.
Limitations
- Evaluated on a single dataset (DroneVehicle) focused on vehicle detection.
- Training is a two-stage procedure and the best way to combine CSOM and ODAF is left for future work.
- Slower than the SLBAF-Net baseline.
Experimental setup
Datasets: DroneVehicle
Benchmarks: DroneVehicle
Metrics: mAP@0.4 fps
Baselines: UA-CMDet Halfway Fusion CIAN AR-CNN MBNet TSFADet C2Former SLBAF-Net RetinaNet Faster R-CNN Oriented R-CNN S2A-Net
Builds on: YOLOv5 SLBAF-Net deformable convolution central moment discrepancy
Closest papers by meaning
- IGIANet: Illumination Guided Implicit Alignment Network for Infrared–Visible UAV Detection (AAAI 2026 · similarity 0.88)
- ABBSPO: Adaptive Bounding Box Scaling and Symmetric Prior based Orientation Prediction for Detecting Aerial Image Objects (CVPR 2025 · similarity 0.85)
- FRED: Towards a Full Rotation-Equivariance in Aerial Image Object Detection (AAAI 2024 · similarity 0.84)
- Alignment-Free RGB-T Salient Object Detection: A Large-Scale Dataset and Progressive Correlation Network (AAAI 2025 · similarity 0.83)
- SSLFusion: Scale and Space Aligned Latent Fusion Model for Multimodal 3D Object Detection (AAAI 2025 · similarity 0.83)
- GSAlign: Geometric and Semantic Alignment Network for Aerial-Ground Person Re-Identification (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.