Research map / Detection and segmentation / CNN features
Spatial Transform Decoupling for Oriented Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spatial Transform Decoupling (STD), which predicts oriented box position, angle, and size with separate ViT decoder stages and guides self-attention with cascaded activation masks, reaching 82.24% mAP on DOTA-v1.0 and 98.55% on HRSC2016.
Problem
Oriented object detection with Vision Transformers is hampered by the lack of spatial transform invariance; direct regression of position, size, and angle from a single feature map yields coupled and suboptimal features.
Approach
Decouple box parameter prediction across stages of a stacked ViT decoder (position first, then angle, then size, then class), and feed cascaded activation masks derived from the predicted boxes into self-attention values to highlight foreground RoI features progressively.
Key results
- STD with HiViT-B in Oriented R-CNN reaches 82.24% mAP on DOTA-v1.0 (multi-scale), versus 81.66% with ViT-B.
- STD-O with ViT-B reaches 98.55% mAP (VOC2012 metric) on HRSC2016 and 90.67% (VOC2007 metric).
- In the single-scale ablation, adding CAMs to the decoupled Faster R-CNN with ViT-S raises mAP from 70.79 to 72.78; Oriented R-CNN improves from 72.41 to 73.43.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MAEBBoxHead with STD | Faster RCNN with ViT-S · DOTA-v1.0 val, single-scale | mAP: 71.07 -> 72.78 | Helped |
| Replace MAEBBoxHead with STD | Oriented RCNN with ViT-S · DOTA-v1.0 val, single-scale | mAP: 72.41 -> 73.43 | Helped |
| Add a pre-trained Transformer head in place of 2FCBBoxHead | Faster RCNN with ViT-small · DOTA-v1.0 | mAP: 69.67 -> 71.07 | Helped |
| Use MAEBBoxHead without pre-training (ablation) | Faster RCNN with ViT-small · DOTA-v1.0 | mAP: 71.07 -> 69.16 | Hurt |
| Add CAMs to decoupled parameter prediction (ablation) | STD · DOTA-v1.0 val, single-scale | mAP: 70.79 -> 72.78 | Helped |
| Use decoupled parameter prediction without CAMs instead of coupled bk3 prediction (ablation) | ViT-small detection head · DOTA-v1.0 val, single-scale | mAP: 70.88 -> 70.79 | Hurt |
| Use the order (xy) → α → (wh) (ablation) | STD parameter prediction · DOTA-v1.0 val, single-scale | mAP: 72.30 -> 72.78 | Helped |
| Apply CAMs exclusively to V rather than Q, K, or their combinations (ablation) | STD self-attention · DOTA-v1.0 val, single-scale | mAP: 72.71 -> 72.78 | Helped |
| Move the auxiliary convolution layers toward shallower parameter branches (ablation) | STD decoupling module · DOTA-v1.0 val, single-scale | mAP: 3, 2, 1: 72.78; 2, 2, 2: 71.96; 1, 2, 3: 71.19 | Mixed |
| Remove the angle prediction branch and retain CAMs as horizontal masks (ablation) | STD-HBB on ViT-small · MS COCO | AP: 44.2 -> 44.2 | No clear effect |
| Add STD to oriented detection | Oriented RCNN with ViT-B · HRSID | mAP (ins.): 54.86 -> 57.59 | Helped |
Surprising findings
- On HRSID, ViT-B outperforms HiViT-B, contrary to the DOTA-v1.0 trend, which the authors attribute to dataset size and overfitting.
- Applying the activation mask only to V (not Q or K) gives the best result in the mask-placement ablation.
Limitations
- Gains on HRSC2016 and HRSID are small or inconsistent across settings, and the paper notes STD gives no clear benefit on general COCO objects.
- Decoupling order and feature-level choices are hand-designed and show sensitivity (e.g., 71.93 to 72.78 mAP across orders).
Experimental setup
Datasets: DOTA-v1.0 HRSC2016 HRSID MS COCO MillionAID
Benchmarks: DOTA-v1.0 HRSC2016 HRSID MS COCO
Metrics: mAP mAP(07) mAP(12) AP
Baselines: RVSA-O KFIoU imTED-O O-RCNN ReDet LSKNet RTMDet-R GWD CSL R3Det
Builds on: Oriented R-CNN Faster R-CNN imTED MAE MMRotate
Closest papers by meaning
- Sub-token ViT Embedding via Stochastic Resonance Transformers (ICML 2024 · similarity 0.85)
- LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition (AAAI 2024 · similarity 0.84)
- ReDiffDet: Rotation-equivariant Diffusion Model for Oriented Object Detection (CVPR 2025 · similarity 0.84)
- SATA: Spatial Autocorrelation Token Analysis for Enhancing the Robustness of Vision Transformers (CVPR 2025 · similarity 0.84)
- Configuring Data Augmentations to Reduce Variance Shift in Positional Embedding of Vision Transformers (AAAI 2025 · similarity 0.84)
- Your ViT is Secretly an Image Segmentation Model (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.