Research map / Detection and segmentation / Open-set and few-shot detection
Discrete Latent Perspective Learning for Segmentation and Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Discrete Latent Perspective Learning (DLPL), a plug-in ViT block that discretizes visual features into latent perspective representations, transforms them via homography-style mapping over an EMA embedding space, and fuses original and transformed perspectives with perspective-invariant attention, improving segmentation and detection on UAV, driving, and daily-photo data.
Problem
Achieving perspective-invariant semantic understanding from single-view images without costly multi-view collection or limited hand-crafted augmentations, especially for UAV imagery with large viewpoint variation.
Approach
Performs multi-perspective fusion in a latent discrete feature space: decomposes features into perspective graphs via discretized levels and a SuperPoint-guided prototype, learns an EMA perspective embedding space, generates transformed perspectives through a homography-like mapping, and fuses them with cross-attention (PIA).
Key results
- DLPL (SegFormer) reaches 59.10% mIoU on DroneSeg versus 52.03% for the SegFormer baseline without augmentation, a gain of about 7 points in the ablation setting.
- DLPL (Deformable DETR) reaches 28.00 AP on VisDrone-dev versus 26.28 for Deformable DETR, and 65.07 AP on UAVDT-test versus 62.99.
- DLPL (SegFormer, MiT-B5) reaches 85.1% mIoU on Cityscapes versus 84.0% for SegFormer MiT-B5, with about 3M added parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DLPL to SegFormer | SegFormer · UDD6 | mIoU (%): 74.54 -> 76.88 | Helped |
| add DLPL to SegFormer | SegFormer · iSAID | mIoU (%): 69.10 -> 70.60 | Helped |
| add DLPL to SegFormer | SegFormer · UAVid | mIoU (%): 66.90 -> 65.30 | Mixed |
| add DLPL to SegFormer | SegFormer · Aeroscapes | mIoU (%): 66.40 -> 68.22 | Helped |
| add DLPL to SegFormer | SegFormer · DroneSeg | mIoU (%): 55.33 -> 59.10 | Helped |
| add DLPL to Sparse DETR | Sparse DETR · VisDrone-dev | AP: 26.98 -> 28.75 | Helped |
| add DLPL to Sparse DETR | Sparse DETR · UAVDT-test | AP: 63.21 -> 65.57 | Helped |
| add DLPL to SegFormer MiT-B5 | SegFormer MiT-B5 · Cityscapes val | mIoU (%): 84.0 -> 85.1 | Helped |
| add DLPL to SegFormer MiT-B5 | SegFormer MiT-B5 · ADE20K val | mIoU (%): 51.8 -> 53.7 | Helped |
| add Random Combination augmentation (ablation) | DLPL with SegFormer · DroneSeg test | mIoU (%): 58.67 -> 59.10 | Helped |
| add AutoAugmentation (ablation) | DLPL with SegFormer · DroneSeg test | mIoU (%): 58.67 -> 59.09 | Helped |
| replace PDD with an ordinary Transformer Network (ablation) | DLPL with SegFormer · DroneSeg test | mIoU (%): 59.10 -> 57.07 | Hurt |
Surprising findings
- Naive perspective augmentations (vertical/horizontal perspective) give only small gains (about 1.4 to 1.5 points), while the combined augmentation reaches 55.33 mIoU, still below DLPL's 58.67 without augmentation.
Limitations
- Perspective transformation is only enabled after 30% of training, and PIA reduces to self-attention before that.
- Ablations are run only on DroneSeg with SegFormer as the baseline.
Experimental setup
Datasets: Cityscapes ADE20K UDD6 iSAID UAVid Aeroscapes DroneSeg VisDrone UAVDT
Benchmarks: Cityscapes ADE20K UDD6 iSAID UAVid Aeroscapes DroneSeg VisDrone UAVDT
Metrics: mIoU AP AP50 AP75
Baselines: SegFormer UperNet SETR PoolFormer Segmenter DETR Deformable DETR Sparse DETR AutoAugment Random Combination augmentation
Builds on: SuperPoint SegFormer Deformable DETR AutoAugment
Closest papers by meaning
- Towards Generalizable Multi-Camera 3D Object Detection via Perspective Rendering (AAAI 2025 · similarity 0.84)
- The 3D-PC: a benchmark for visual perspective taking in humans and machines (ICLR 2025 · similarity 0.84)
- Multi-view Aggregation Network for Dichotomous Image Segmentation (CVPR 2024 · similarity 0.83)
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.83)
- DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection (AAAI 2024 · similarity 0.83)
- DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.