Research map / Label-efficient and robust learning / Recognition and long-tail
FaceCoresetNet: Differentiable Coresets for Face Set Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FaceCoresetNet, which selects a small differentiable, quality-aware coreset of a face template via Gumbel-Softmax farthest-point sampling, then applies self/cross-attention for linear-time set face recognition.
Problem
Set-based face recognition requires aggregating an unbounded, unordered set of face images and videos into one discriminative descriptor, balancing image quality and diversity (face burstiness) efficiently and permutation-invariantly; prior attention methods are quadratic in template size.
Approach
Replaces non-differentiable argmax in farthest-point sampling with Gumbel-Softmax sampling over a quality-aware cosine distance (scaled by feature norm raised to a learned exponent gamma), so the coreset selection is trained end-to-end; the small coreset (K=3) then serves as queries for self- and cross-attention over the full template, giving O(N) complexity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FaceCoresetNet instead of CAFace | Face verification · IJB-B 1:1 Verification | TAR@FAR=1e-4: 95.78 -> 95.28 | Hurt |
| use FaceCoresetNet instead of CAFace | Face verification · IJB-B 1:1 Verification | TAR@FAR=1e-5: 92.78 -> 91.44 | Hurt |
| use FaceCoresetNet instead of SD + VBA | Face verification · IJB-B 1:1 Verification | TAR@FAR=1e-6: 48.96 -> 52.56 | Helped |
| use FaceCoresetNet instead of CAFace | Face verification · IJB-C 1:1 Verification | TAR@FAR=1e-4: 97.30 -> 96.96 | Hurt |
| use FaceCoresetNet instead of CAFace | Face verification · IJB-C 1:1 Verification | TAR@FAR=1e-5: 95.96 -> 95.21 | Hurt |
| use FaceCoresetNet instead of CAFace | Face verification · IJB-C 1:1 Verification | TAR@FAR=1e-6: 90.56 -> 91.12 | Helped |
| add differential core-template selection to average pooling (ablation) | FaceCoresetNet · IJB-B | TAR@FAR=1e-6: 38.45 -> 48.01 | Helped |
| add self-attention to core-template features (ablation) | FaceCoresetNet with core-template selection · IJB-B | TAR@FAR=1e-6: 48.01 -> 49.24 | Helped |
| add cross-attention between the core and full templates (ablation) | FaceCoresetNet with core-template self-attention · IJB-B | TAR@FAR=1e-6: 49.24 -> 51.71 | Helped |
| add norm encoding (ablation) | FaceCoresetNet with self- and cross-attention · IJB-B | TAR@FAR=1e-6: 51.71 -> 52.56 | Helped |
| train γ end-to-end instead of fixing it at 0 (ablation) | FaceCoresetNet · IJB-C | TAR@FAR=1e-6: 90.84 -> 91.12 | Helped |
| train γ end-to-end instead of fixing it at 10 (ablation) | FaceCoresetNet · IJB-C | TAR@FAR=1e-6: 89.69 -> 91.12 | Helped |
Surprising findings
- Selecting only 3 core-template features and averaging them raises IJB-B TAR@FAR=1e-6 substantially over averaging the full template (48.01 vs 38.45).
- FaceCoresetNet is second-best or lower at FAR=1e-4 and 1e-5 on IJB-B, trailing CAFace, so the gains are concentrated in the lowest FAR regime.
Limitations
- Gains are concentrated at the very low FAR regime (1e-6); at FAR=1e-4 and 1e-5 it trails CAFace on IJB-B.
- Evaluation is limited to face verification on IJB-B and IJB-C; identification is not reported in the text.
- Feed-forward blocks were removed from the attention encoder/decoder because they degraded performance, a design choice not deeply analyzed.
Experimental setup
Datasets: WebFace4M IJB-B IJB-C
Benchmarks: IJB-B IJB-C
Metrics: TAR@FAR=1e-4 TAR@FAR=1e-5 TAR@FAR=1e-6
Baselines: AdaFace RSA SD + VBA CAFace
Builds on: AdaFace CAFace Gumbel-Softmax Efficient-DETR
Closest papers by meaning
- Quality Aware Network for Set to Set Recognition (CVPR 2017 · similarity 0.88)
- AdaptiveFace: Adaptive Margin and Sampling for Face Recognition (CVPR 2019 · similarity 0.87)
- Discriminative Covariance Oriented Representation Learning for Face Recognition With Image Sets (CVPR 2017 · similarity 0.87)
- Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.87)
- RepFace: Refining Closed-Set Noise with Progressive Label Correction for Face Recognition (AAAI 2025 · similarity 0.84)
- HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.