Research map / Architectures and efficiency / RNNs and normalization
Path-Adaptive Matting for Efficient Inference Under Various Computational Cost Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Path-Adaptive Matting (PAM), a single matting network that adapts its inference path to image content and a FLOP budget, trained via bilevel optimization with performance-aware online path labeling.
Problem
Efficient image matting under varying computational (FLOP) cost constraints, since existing matting networks are non-scalable and need retraining for each budget.
Approach
A path-adaptive architecture with per-layer path selection layers (channel-attention-based, conditioned on a cost-constraint embedding) and learnable pointwise connect layers that approximate bypassed layers, trained with online path labels chosen among prior-sampled and network-predicted paths by evaluating their matting error.
Key results
- PAM (Moderate) reports SAD 23.25 on Adobe Composition-1K with 69.6 GFLOPs and 7.1M parameters, versus MatteFormer's 23.80 SAD at 233.3 GFLOPs and 44.9M parameters.
- PAM (Moderate) uses about 4.5% of LFPNet's computation and 6.3% of its parameters while performing comparably.
- PAM's Mild, Moderate and Aggressive variants all run in 8 GB peak memory on an R9 3900X CPU for 2048x2048 inputs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relax the cost constraint from aggressive to mild | PAM · Adobe Composition-1K | SAD: 24.05 -> 23.14 | Helped |
| Relax the cost constraint from aggressive to mild | PAM · Distinctions-646 | SAD: 22.97 -> 21.97 | Helped |
| Relax the cost constraint from aggressive to mild | PAM · Transparent-460 | SAD: 210.11 -> 194.09 | Helped |
| Relax the cost constraint from aggressive to mild | PAM · Semantic Image Matting | SAD: 25.37 -> 24.39 | Helped |
| Tighten the cost constraint from mild to aggressive | PAM · Automatic Image Matting-500 | SAD: 25.14 -> 24.35 | Helped |
Surprising findings
- Under the aggressive constraint PAM outperforms the mild setting on AIM-500 (SAD 24.35 vs 25.14), which the authors attribute to domain shift from synthetic training data.
Limitations
- Performance drops under aggressive FLOP constraints relative to heavier baselines, and gap to best methods such as VitMatte remains on Adobe Composition-1K.
- Trained only on synthetic Adobe Composition-1K data; generalization to real-world images shows a domain-shift effect.
Experimental setup
Datasets: Adobe Composition-1K Distinctions-646 Transparent-460 Semantic Image Matting (SIMD) Automatic Image Matting-500 (AIM-500)
Benchmarks: Adobe Composition-1K Distinctions-646 Transparent-460 Semantic Image Matting AIM-500
Metrics: SAD MSE GRAD CONN FLOPs parameters latency memory
Baselines: DIM IndexNet GCAMatting FBAMatting SIM TIMI LFPNet MatteFormer DiffusionMat VitMatte TransMatting MGMatting
Builds on: Gumbel-Softmax SPOS MatteFormer LASNet
Closest papers by meaning
- Memory Efficient Matting with Adaptive Token Routing (AAAI 2025 · similarity 0.87)
- Training Matting Models Without Alpha Labels (AAAI 2025 · similarity 0.87)
- Revisiting Context Aggregation for Image Matting (ICML 2024 · similarity 0.86)
- MatAnyone: Stable Video Matting with Consistent Memory Propagation (CVPR 2025 · similarity 0.85)
- MaGGIe: Masked Guided Gradual Human Instance Matting (CVPR 2024 · similarity 0.85)
- Privileged Prior Information Distillation for Image Matting (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.