Research map / Label-efficient and robust learning / Recognition and long-tail
Training Matting Models Without Alpha Labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains image matting models using only trimaps as labels by combining an l1 known-region loss with a directional distance consistency (DDC) loss that propagates alpha values from known to unknown regions via image-guided nonlocal similarity.
Problem
Deep image matting requires expensive fine alpha matte annotations, which limits dataset scale; the paper asks whether models can be trained from coarse trimap labels alone.
Approach
A directional distance consistency loss enforces that pairwise alpha differences between similar pixel pairs match their image distances (with sign preserved), applying a nonlocal matting prior as a training loss so unknown transition alphas are inferred from learned known regions, without alpha labels.
Key results
- On AM-2K, the trimap-supervised model achieves SAD 30.26 and MSE 0.0099, versus SAD 26.00 and MSE 0.0101 for the matte-supervised baseline, with better Grad (14.98 vs 16.04).
- On AM-2K, the trimap-supervised model is comparable to recent automatic matting methods but behind GFM (SAD 10.26).
- On P3M-NP-500, the trimap-supervised model gets SAD 31.66 versus 11.23 for P3M with matte supervision, so it lags behind fully supervised methods on portrait data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with trimap labels and known loss plus DDC loss instead of fine alpha matte labels | ViTMatte-S · Training paradigm | SAD: 26.00 -> 30.26 | Hurt |
| Train with trimap labels and known loss plus DDC loss instead of fine alpha matte labels | ViTMatte-S · Training paradigm | MSE: 0.0101 -> 0.0099 | Helped |
| Add affinity loss to known loss (ablation) | Trimap-supervised ViTMatte-S · AM-2K test | SAD: 45.94 -> 32.54 | Helped |
| Replace affinity loss with DC loss (ablation) | Known-loss trimap training · AM-2K test | SAD: 32.54 -> 42.24 | Hurt |
| Replace DC loss with DDC loss (ablation) | Known-loss trimap training · AM-2K test | SAD: 42.24 -> 30.26 | Helped |
| Replace DC loss with DDC loss (ablation) | Known-loss trimap training · AM-2K test | Grad: 16.07 -> 14.98 | Helped |
| Restrict DDC loss from the whole alpha map to unknown regions (ablation) | Known-loss trimap training · AM-2K test | SAD: 30.26 -> 38.80 | Hurt |
| Restrict DC loss from the whole alpha map to unknown regions (ablation) | Known-loss trimap training · AM-2K test | SAD: 42.24 -> 72.62 | Hurt |
| Increase DDC window size from 5 to 11 (ablation) | Trimap-supervised ViTMatte-S · AM-2K test | SAD: 41.01 -> 30.26 | Helped |
| Increase DDC window size from 11 to 13 (ablation) | Trimap-supervised ViTMatte-S · AM-2K test | SAD: 30.26 -> 33.01 | Hurt |
| Use fixed trimap erosion size 30 instead of random size Rand(1,30) (ablation) | DDC training · AM-2K test | SAD: 30.26 -> 44.48 | Hurt |
| Replace trimap labels with segmentation masks and l1 known loss (ablation) | DDC training · AM-2K test | SAD: 30.26 -> 40.47 | Hurt |
Surprising findings
- The trimap-supervised model slightly beats the matte-supervised baseline on MSE and Grad on AM-2K.
- Using DC loss without the sign-preserving modification increases SAD error due to texture noise in the interior regions.
Limitations
- Performance on P3M-NP-500 portrait matting is substantially worse than fine-label and recent automatic matting baselines.
- Only automatic matting is evaluated; other matting modes are left to future work.
- Trimaps are generated synthetically from alpha erosion rather than by human annotators.
Experimental setup
Datasets: AM-2K P3M-10K
Benchmarks: AM-2K P3M-NP-500
Metrics: SAD MAD MSE Grad Conn SAD-T MSE-T
Baselines: SHM LF SSS HATT GFM MODNet P3M ViTMatte-S fine-label baseline
Builds on: ViTMatte KNN Matting Nonlocal Matting Deep Image Matting
Closest papers by meaning
- Image Matting without Matting-Specific Annotations via Eikonal Fields (NeurIPS 2026 · similarity 0.92)
- Boosting Robustness of Image Matting With Context Assembling and Strong Data Augmentation (CVPR 2022 · similarity 0.91)
- In-Context Matting (CVPR 2024 · similarity 0.90)
- Revisiting Context Aggregation for Image Matting (ICML 2024 · similarity 0.89)
- Privileged Prior Information Distillation for Image Matting (AAAI 2024 · similarity 0.88)
- MaGGIe: Masked Guided Gradual Human Instance Matting (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.