Research map / Detection and segmentation / Open-set and few-shot detection

SegFace: Face Segmentation of Long-Tail Classes

AAAI 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Interactive and referring segmentation · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SegFace, a lightweight transformer decoder with learnable class-specific tokens over multi-scale Swin features, to improve face parsing especially for long-tail accessory classes.

Problem

Face parsing models, mostly CNN-based, are dominated by frequent head classes and perform poorly on infrequent long-tail classes such as eyeglasses, earrings, necklaces and hats.

Approach

Assign each semantic class a learnable token in a lightweight transformer decoder so each token independently models its class via cross-attention to multi-scale fused face features; output masks are the product of upscaled face features and per-class MLP-transformed tokens.

Key results

• On CelebAMask-HQ, SegFace (Swin, 512 res) reaches 88.96 mean F1 and 81.55 mean IoU, versus 86.07 for DML-CSR (table-reported baseline).
• On LaPa, SegFace at 448/512 resolution reaches 93.03 mean F1, versus 92.38 for DML-CSR and FP-LIIF.
• A MobileNetV3-backbone variant reaches 87.91 mean F1 at 95.96 FPS on CelebAMask-HQ.

Measured effects reported in the paper

Surprising findings

• On the small Helen dataset (2,000 training images), SegFace underperforms the DML-CSR baseline (91.8 vs 93.8 mean F1), which the authors attribute to the transformer decoder needing more data.
• Backbone choice matters little: backbones from 7M to 91M parameters give similar mean F1, which the authors take as evidence the decoder does most of the work.

Limitations

• Transformer decoder underperforms on scarce training data (Helen, 2,000 samples).
• Gains on some long-tail classes remain modest (e.g., eyeglasses, hat lower-level F1).

Experimental setup

Datasets: LaPa CelebAMask-HQ Helen

Benchmarks: LaPa CelebAMask-HQ Helen

Metrics: mean F1 mean IoU class-wise F1 FPS

Baselines: Wei et al. BASS EAGRNet AGRNet FaRL DML-CSR FP-LIIF

Builds on: Mask2Former Swin Transformer DML-CSR FaRL

Closest papers by meaning

• A New Dataset and Boundary-Attention Semantic Segmentation for Face Parsing (AAAI 2020 · similarity 0.89)
• Face Parsing With RoI Tanh-Warping (CVPR 2019 · similarity 0.88)
• GroundingFace: Fine-grained Face Understanding via Pixel Grounding Multimodal Large Language Model (CVPR 2025 · similarity 0.85)
• Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective (NeurIPS 2024 · similarity 0.85)
• HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.85)
• $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.85)