Research map / Video / Video segmentation and tracking
Polarized Color Screen Matting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Polarized Color Screen Matting (pCSM), a single-shot per-pixel closed-form matting theory combining color and polarization cues on a polarized known-color LCD backdrop, with pCSM-S and pCSM-DS variants.
Problem
Alpha matte extraction from video against a known background is ill-posed when using only color (chroma keying) or only polarization (polarimetric matting), especially when foreground reflections are polarized or color-similar to the background.
Approach
A diffuse-specular polarimetric compositing equation that models foreground as diffuse plus specular reflection and jointly uses color and polarization observations from a polarization camera, yielding per-pixel closed-form solutions with solvability conditions, plus a polarized-diffuse ridge regularization for robustness to noise.
Key results
- On the PolarMatte static scene test set, regularized non-linear pCSM-DS achieves MAE 7.48 vs 10.35 for PolarMatte and 27.13 for Keylight.
- pCSM-S runs in 2.28 ms per 2K image on GPU, over 9 times faster than BGMv2 (20.55 ms).
- Non-linear pCSM-DS recovers the noise-free synthetic alpha matte with zero error on datasets A and B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine color and polarization cues in pCSM-S | Polarized screen matting · PolarMatte static scene test data | MAE: 11.26 -> 7.57 | Helped |
| use regularized non-linear pCSM-DS instead of pCSM-S | pCSM-S · PolarMatte static scene test data | MAE: 7.57 -> 7.48 | Helped |
| add polarized diffuse regularization (ablation) | Non-linear pCSM-DS · Synthetic datasets A/B, noisy | MAE: 23.28 37.48 -> 17.78 19.65 | Helped |
| add polarized diffuse regularization (ablation) | Non-linear pCSM-DS · Synthetic datasets C/D, noisy | MAE: 24.19 25.82 vs NA NA | Hurt |
| add polarized diffuse regularization (ablation) | Non-linear pCSM-DS · Synthetic datasets A/B, noise-free | MAE: 0.00 0.00 -> 3.94 5.74 | Hurt |
| use pCSM-S instead of PTM | Polarized screen matting · 2K images | processing time: 1.80 ms -> 2.28 ms | Hurt |
| use pCSM-S instead of BGMv2 | Matting · 2K images | processing time: 20.55 ms -> 2.28 ms | Helped |
Surprising findings
- Keylight, a commercial tool operated by an expert, yields lower accuracy than polarization-based PTM and PolarMatte on the static test set.
- BGMv2 obtains a very high MAE (85.20) on the static green-screen scenes, far worse than the other methods.
Limitations
- Non-linear pCSM-DS is much slower (tens of seconds per frame on CPU) and not real-time.
- Unregularized pCSM-DS is sensitive to noise; closed-form solvability fails for strong or weak color/polarization conditions in some synthetic cases (C, D).
- Assumes known background, known illumination color, and a diffuse-specular reflectance model; real-world evaluation relies on a single static test set with reference mattes from triangulation matting.
Experimental setup
Datasets: PolarMatte static scene test data (12 scenes) synthetic datasets A-D (1000 pixels each)
Benchmarks: PolarMatte static scene test data synthetic datasets A-D
Metrics: MAE MSE GRAD processing time
Baselines: Keylight PTM BGMv2 PolarMatte triangulation matting
Builds on: PolarMatte polarimetric triangulation matting dichromatic reflectance model triangulation matting
Closest papers by meaning
- PolarMatte: Fully Computational Ground-Truth-Quality Alpha Matte Extraction for Images and Video using Polarized Screen Matting (CVPR 2024 · similarity 0.94)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.85)
- DRIP: Unleashing Diffusion Priors for Joint Foreground and Alpha Prediction in Image Matting (NeurIPS 2024 · similarity 0.85)
- In-Context Matting (CVPR 2024 · similarity 0.84)
- Learning to Remove Wrinkled Transparent Film with Polarized Prior (CVPR 2024 · similarity 0.84)
- PolarFree: Polarization-based Reflection-Free Imaging (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.