Research map / Architectures and efficiency / Attention and transformers
Revisiting Context Aggregation for Image Matting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits context aggregation in image matting, finds plain encoder-decoder nets learn context well, and proposes AEMatter (Hybrid-Transformer backbone with appearance-enhanced axis-wise learning blocks, trained on 1024x1024 patches) that outperforms SOTA on five matting datasets.
Problem
Pooling- and affinity-based context aggregation modules in deep image matting are sensitive to context scale shift between training and inference image sizes, degrading matting accuracy.
Approach
A basic encoder-decoder without explicit context aggregation modules can learn universal context aggregation when trained on large patches with large receptive fields; AEMatter combines a Hybrid-Transformer (conv stem + Swin) encoder with appearance-enhanced axis-wise attention blocks to enlarge the receptive field.
Key results
- On Adobe Composition-1K, AEMatter achieves SAD 17.53, MSE 2.26, Grad 4.76, Conn 12.46 versus 20.33 SAD for ViTMatte-B; with TTA SAD is 16.89.
- On Transparent-460 generalization, AEMatter reaches SAD 122.27 versus 192.36 for TransMatting.
- On Distinctions-646 generalization, AEMatter reaches SAD 16.95 versus 22.36 for LFPNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the basic ResNet-34 network with larger patches (256 to 1024) (ablation) | BasicNet, ResNet-34 · Adobe Composition-1K validation | SAD: 41.74 -> 28.08 | Helped |
| Train the basic Swin-Tiny network with larger patches (256 to 1024) (ablation) | BasicNet, Swin-Tiny · Adobe Composition-1K validation | SAD: 27.99 -> 19.72 | Helped |
| Replace 3 × 3 convolutions with 5 × 5 convolutions (ablation) | BasicNet, ResNet-34 · Adobe Composition-1K validation | SAD: 28.08 -> 26.72 | Helped |
| Replace 3 × 3 convolutions with 1 × 1 convolutions (ablation) | BasicNet, ResNet-34 · Adobe Composition-1K validation | SAD: 28.08 -> 31.28 | Hurt |
| Replace 3 × 3 convolutions with 5 × 5 convolutions (ablation) | BasicNet, ResNet-50 · Adobe Composition-1K validation | SAD: 23.82 -> 23.34 | Helped |
| Replace 3 × 3 convolutions with 1 × 1 convolutions (ablation) | BasicNet, ResNet-50 · Adobe Composition-1K validation | SAD: 23.82 -> 28.70 | Hurt |
| Add test-time augmentation | AEMatter · Adobe Composition-1K | SAD: 17.53 -> 16.89 | Helped |
| Use AEMatter instead of the strongest reported baseline | Matting methods · Adobe Composition-1K | SAD: 20.33 -> 17.53 | Helped |
| Use AEMatter instead of the strongest reported baseline | Matting methods · Distinctions-646 | SAD: 22.36 -> 16.95 | Helped |
| Use AEMatter instead of the strongest reported baseline | Matting methods · Transparent-460 | SAD: 192.36 -> 122.27 | Helped |
| Use AEMatter instead of the strongest reported baseline | Matting methods · Semantic Image Matting | SAD: 23.05 -> 19.51 | Helped |
| Use AEMatter instead of the strongest reported baseline | Matting methods · Automatic Image Matting-500 | SAD: 19.43 -> 14.76 | Helped |
Surprising findings
- Existing matting networks with context aggregation modules show error that first decreases then increases or stabilizes as inference patch size grows, while the plain IndexNet error decreases monotonically.
- A basic encoder-decoder without context aggregation modules (BasicNet) outperforms several state-of-the-art matting methods with such modules.
Limitations
- Context aggregation modules with hand-crafted designs are sensitive to context scale, which the paper argues limits generalization.
- Training on 1024x1024 patches is larger than prior methods, implying higher training cost not quantified in the text.
Experimental setup
Datasets: Adobe Composition-1K Distinctions-646 Transparent-460 Semantic Image Matting Automatic Image Matting-500
Benchmarks: Adobe Composition-1K Distinctions-646 Transparent-460 Semantic Image Matting Automatic Image Matting-500
Metrics: SAD MSE Grad Conn
Baselines: IndexNet GCAMatting TIMI-Net FBAMatting MatteFormer ViTMatte-B DiffMatte-ViTS dugMatting DiffusionMat TransMatting LFPNet MGMatting
Builds on: IndexNet FBAMatting MatteFormer Swin Transformer TIMI-Net
Closest papers by meaning
- Attention-Guided Hierarchical Structure Aggregation for Image Matting (CVPR 2020 · similarity 0.90)
- In-Context Matting (CVPR 2024 · similarity 0.90)
- Training Matting Models Without Alpha Labels (AAAI 2025 · similarity 0.89)
- Unifying Automatic and Interactive Matting with Pretrained ViTs (CVPR 2024 · similarity 0.87)
- Memory Efficient Matting with Adaptive Token Routing (AAAI 2025 · similarity 0.87)
- MatAnyone: Stable Video Matting with Consistent Memory Propagation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.