Research map / Detection and segmentation / CNN features
Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BPT, a single-branch ultra-high-resolution segmentation transformer that dynamically merges patch tokens (PMT) and uses training-only boundary supervision (BEM) to beat prior UHR methods on five benchmarks without extra inference cost.
Problem
Semantic segmentation of ultra-high-resolution images, where dual-branch global/local designs struggle to fuse global and local information and add substantial memory and compute cost.
Approach
A Patch-Merging Transformer clusters patch features with a DPC-KNN density-peak scheme so informative regions get fine tokens and homogeneous regions get coarse tokens, combined with an auxiliary boundary-prediction branch used only during training and a feature fusion module to avoid conflicts between global and boundary features.
Key results
- On DeepGlobe, BPT reaches 76.6% mIoU, above GPWFormer (75.8%) and GeoAgent (75.4%) with 2074 MB memory.
- On Inria Aerial, BPT reaches 77.1% mIoU versus 76.5% for GPWFormer.
- On Cityscapes, BPT reaches 78.5% mIoU versus 78.1% for GPWFormer; on CRAG it reaches 90.9% mIoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BPT rather than GPWFormer | UHR image segmentation · DeepGlobe test | mIoU (%): 75.8 -> 76.6 | Helped |
| Use BPT rather than GPWFormer | UHR image segmentation · Inria Aerial test | mIoU (%): 76.5 -> 77.1 | Helped |
| Use BPT rather than GPWFormer | UHR image segmentation · Cityscapes | mIoU (%): 78.1 -> 78.5 | Helped |
| Use BPT rather than GPWFormer | UHR image segmentation · ISIC | mIoU (%): 80.7 -> 81.6 | Helped |
| Use BPT rather than GPWFormer | UHR image segmentation · CRAG | mIoU (%): 89.9 -> 90.9 | Helped |
| Add PMB and PRB (ablation) | BPT · DeepGlobe | mIoU (%): 73.9 -> 76.6 | Helped |
| Add PMB and PRB (ablation) | BPT · Inria Aerial | mIoU (%): 74.8 -> 77.1 | Helped |
| Add PMB and PRB (ablation) | BPT · Cityscapes | mIoU (%): 76.9 -> 78.5 | Helped |
| Use PMB instead of convolutional downsampling (ablation) | BPT · DeepGlobe | mIoU (%): 75.0 -> 76.6 | Helped |
| Use PMB instead of convolutional downsampling (ablation) | BPT · Inria Aerial | mIoU (%): 75.8 -> 77.1 | Helped |
| Use PMB instead of convolutional downsampling (ablation) | BPT · Cityscapes | mIoU (%): 77.7 -> 78.5 | Helped |
| Use PRB instead of a deconvolutional head (ablation) | BPT · DeepGlobe | mIoU (%): 75.5 -> 76.6 | Helped |
Limitations
- Gains over the strongest prior method (GPWFormer) are small, about 0.5-0.8 mIoU points.
- Memory on DeepGlobe and Cityscapes is higher than several generic and UHR baselines, so the efficiency claim is relative rather than absolute.
- Baseline numbers are copied from prior papers rather than re-run under identical settings.
Experimental setup
Datasets: ImageNet-1K DeepGlobe Inria Aerial Cityscapes ISIC CRAG
Benchmarks: DeepGlobe Inria Aerial Cityscapes ISIC CRAG
Metrics: mIoU F1 pixel accuracy GPU memory
Baselines: U-Net PSPNet DeepLabv3+ FCN-8s STDC CascadePSP GLNet ISDNet FCtL WSDNet GeoAgent GPWFormer
Builds on: PVT DPC-KNN GPWFormer GLNet boundary relaxation loss
Closest papers by meaning
- Toward Real Ultra Image Segmentation: Leveraging Surrounding Context to Cultivate General Segmentation Model (NeurIPS 2024 · similarity 0.87)
- Boosting the Dual-Stream Architecture in Ultra-High Resolution Segmentation with Resolution-Biased Uncertainty Estimation (CVPR 2025 · similarity 0.86)
- High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity (ICLR 2025 · similarity 0.84)
- UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset (NeurIPS 2025 · similarity 0.84)
- xT: Nested Tokenization for Larger Context in Large Images (ICML 2024 · similarity 0.83)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.