Research map / Detection and segmentation / CNN features
Multi-view Aggregation Network for Dichotomous Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models dichotomous image segmentation as multi-view perception, using a single-stream single-encoder-decoder network (MVANet) that fuses a distant low-resolution view and close-up high-resolution patches via cross-attention localization and refinement modules.
Problem
High-accuracy dichotomous image segmentation (DIS) of high-resolution images must balance global semantic localization against fine detail preservation, while existing multi-stream, multi-stage methods are complex and slow.
Approach
Splits a high-resolution image into a downsampled distant view plus non-overlapping close-up patches processed in one batch through a single encoder-decoder; multi-view complementary localization (MCLM) and refinement (MCRM) modules use cross-attention with multi-granularity pooled global tokens to exchange information between views, followed by a simple view rearrangement head.
Key results
- On DIS-TE1 to TE4 overall, MVANet reports F-max 0.916 and MAE 0.035 on DIS5K, outperforming the second-best InSPyReNet by about 2.5% in F-max and 2.1% in E-measure.
- MVANet runs at 4.6 FPS versus 2.2 FPS for InSPyReNet on an RTX 3090.
- Using 4 close-up patches is best; 9 and 16 patches degrade F-max from 0.893 to 0.821 and 0.717 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MVANet instead of InSPyReNet | DIS5K segmentation with Swin-B · DIS5K Overall | F max β: 0.891 -> 0.916 | Helped |
| Use MVANet instead of InSPyReNet | DIS5K segmentation with Swin-B · DIS5K Overall | MAE: 0.039 -> 0.035 | Helped |
| Combine distant-view images and close-up patches (ablation) | MVANet input strategy · DIS-TE1 | F max β: 0.822 -> 0.893 | Helped |
| Add the distant-view image to HR-Ori (ablation) | MVANet input strategy · DIS-TE1 | F max β: 0.822 -> 0.875 | Helped |
| Add close-up patches to the distant-view image (ablation) | MVANet input strategy · DIS-TE1 | F max β: 0.815 -> 0.893 | Helped |
| Add the multi-view complementary localization module (MCLM) (ablation) | MVANet · DIS-TE1 | F max β | Helped |
| Add the multi-view complementary refinement module (MCRM) (ablation) | MVANet · DIS-TE1 | F max β | Helped |
| Add the view rearrangement module (VRM) (ablation) | MVANet · DIS-TE1 | F max β | Helped |
| Increase the close-up patch count from 4 to 9 (ablation) | MVANet · DIS-TE1 | F max β: 0.893 -> 0.821 | Hurt |
| Increase the close-up patch count from 4 to 16 (ablation) | MVANet · DIS-TE1 | F max β: 0.893 -> 0.717 | Hurt |
Surprising findings
- Increasing the number of close-up patches from 4 to 16 substantially degrades performance, since per-patch resolution and global-view resolution both drop.
Limitations
- Performance degrades sharply as the number of close-up patches grows beyond four.
- Patch-boundary misalignment requires an extra convolutional rearrangement head.
Experimental setup
Datasets: DIS5K
Benchmarks: DIS5K (DIS-TE1-4)
Metrics: max F-measure weighted F-measure E-measure S-measure MAE FPS
Baselines: F3Net GCPANet PFNet BSANet ISDNet IFA IS-Net FP-DIS UDUN PGNet InSPyReNet
Builds on: Swin Transformer FPN IS-Net UDUN
Closest papers by meaning
- Revisiting the Integration of Convolution and Attention for Vision Backbone (NeurIPS 2024 · similarity 0.85)
- MaskFactory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation (NeurIPS 2024 · similarity 0.85)
- High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity (ICLR 2025 · similarity 0.85)
- Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.85)
- OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation (NeurIPS 2025 · similarity 0.85)
- DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.