Research map / Video / Video segmentation and tracking
PolarMatte: Fully Computational Ground-Truth-Quality Alpha Matte Extraction for Images and Video using Polarized Screen Matting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PolarMatte, a fully computational alpha matte extraction method that uses a single polarization camera and a polarized LCD backlight to produce ground-truth-quality mattes for images and video without manual correction.
Problem
Creating high-quality ground-truth alpha mattes for video matting datasets is laborious, requiring manual correction or constrained capture; existing backlit methods (triangulation matting, polarimetric triangulation matting) fail when foreground reflects polarized light.
Approach
Model polarized foreground reflections explicitly with a polarimetric compositing equation using four polarization states, detect well-posed pixels with a robust Bayesian least-squares estimate and phase-angle thresholding, and propagate alpha with a novel polarimetric matting Laplacian plus a PolarIC intensity correction.
Key results
- PolarMatte achieves MAE 7.65, MSE 0.28, GRAD 0.72 on 12 static test scenes, versus PTM 8.38/0.33/1.03, BGM 137.75/110.43/21.96, and Keylight 84.45/84.45/6.13.
- Replacing the polarimetric matting Laplacian with the original matting Laplacian raises MAE from 7.65 to 9.89 and GRAD from 0.72 to 2.22.
- Processing takes several tens of seconds per image on a Xeon CPU, versus about one second for PTM, so it is unsuitable for real-time use.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PolarMatte instead of PTM | Alpha matte extraction · Static scene test data | MAE: 8.38 -> 7.65 | Helped |
| use PolarMatte instead of PTM | Alpha matte extraction · Static scene test data | MSE: 0.33 -> 0.28 | Helped |
| use PolarMatte instead of PTM | Alpha matte extraction · Static scene test data | GRAD: 1.03 -> 0.72 | Helped |
| add PolarIC adjustment (ablation) | PolarMatte · Static scene test data | MAE: 8.78 -> 7.65 | Helped |
| add Bayesian framework (ablation) | PolarMatte · Static scene test data | MAE: 8.53 -> 7.65 | Helped |
| add phase angle thresholding (ablation) | PolarMatte · Static scene test data | MAE: 8.85 -> 7.65 | Helped |
| add polarimetric matting Laplacian (ablation) | PolarMatte · Static scene test data | GRAD: 1.04 -> 0.72 | Helped |
| replace original matting Laplacian with polarimetric matting Laplacian (ablation) | PolarMatte · Static scene test data | MAE: 9.89 -> 7.65 | Helped |
Surprising findings
- Keylight, a commercial chroma keying tool, has a much larger MAE (84.45) than the classic PTM baseline, and BGM performs far worse than all others on static scenes.
Limitations
- Incorrect fractional alpha when polarized foreground reflection resembles the polarized backlight, especially when subject is close to the screen.
- High computational cost (tens of seconds per image) makes real-time background replacement unrealistic.
- Evaluated on 12 static scenes for quantitative results; dynamic results are qualitative only.
Experimental setup
Benchmarks: in-house static scene test data (12 scenes)
Metrics: MAE MSE GRAD
Baselines: PTM BGM Keylight triangulation matting original matting Laplacian
Builds on: polarimetric triangulation matting triangulation matting matting Laplacian background matting
Closest papers by meaning
- Polarized Color Screen Matting (CVPR 2025 · similarity 0.94)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.86)
- DRIP: Unleashing Diffusion Priors for Joint Foreground and Alpha Prediction in Image Matting (NeurIPS 2024 · similarity 0.84)
- PolarFree: Polarization-based Reflection-Free Imaging (CVPR 2025 · similarity 0.84)
- Training Matting Models Without Alpha Labels (AAAI 2025 · similarity 0.84)
- PolarScale: A Physics-Grounded Benchmark for Radiometrically Consistent RGB-to-Stokes Estimation (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.