Research map / Federated learning and privacy / Privacy attacks and secure learning
Privacy-Preserving Optics for Enhancing Protection in Face De-Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hardware-level face de-identification pipeline that jointly learns a phase-mask optical encoder and heatmap regressor, then uses a StarGAN2-style GAN to synthesize a new face from the privacy-preserving image, heatmap, and a reference face.
Problem
Software-level face de-identification assumes access to high-fidelity source images, leaving a security gap where adversaries can sniff raw captures before processing; the paper targets privacy at image acquisition while keeping utility for downstream face tasks.
Approach
Learns Zernike-parameterized phase-mask optics that filter high frequencies to conceal identity while preserving face geometry for heatmap regression, and conditions a StarGAN2-based generator on the optically encoded image, heatmap, and a reference style, with a reference-based discriminator and added expression-consistency and LPIPS losses.
Key results
- On CelebA-HQ, the learned-optics method achieves the best VGGFace2 DIS (1.251 vs 1.224 for the best baseline) and FID 29.218, comparable to baselines.
- Under DDRM deconvolution, the learned lens yields PSNR of about 12-16 dB versus about 22-24 dB for low-resolution camera images, indicating the identity was not recovered.
- In a 106-person human study, 91.82% of participants failed to identify the source face among six candidates in the de-identified scenario.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the face heatmap to the framework (ablation) | Face de-identification framework · CelebA-HQ | DIS (FR): 0.851 -> 0.847 | No clear effect |
| Add the face heatmap to the framework (ablation) | Face de-identification framework · CelebA-HQ | Landmarks (MtCNN): 5.434 -> 5.309 | Helped |
| Add LPIPS loss to the heatmap-based framework (ablation) | Face de-identification framework · CelebA-HQ | FID: 36.056 -> 29.338 | Helped |
| Add expression consistency loss to the heatmap-based framework (ablation) | Face de-identification framework · CelebA-HQ | FID: 36.056 -> 24.998 | Helped |
| Use the full framework with heatmap, LPIPS, and expression consistency losses (ablation) | Face de-identification framework · CelebA-HQ | FID: 36.056 -> 24.972 | Helped |
| Use a low-resolution camera instead of the learned lens | Face de-identification framework · CelebA-HQ | DIS (VGGFace2): 1.251 -> 1.294 | Helped |
| Use the optimized lens instead of a low-resolution camera | Privacy-preserving camera · Deconvolution attack | PSNR: 23.75 -> 12.02 | Helped |
| Use the optimized lens rather than low-resolution capture | Privacy-preserving camera · Human evaluation | Highly Protected responses: 12.76% -> 56.00% | Helped |
| Generate a de-identified face with the proposed framework | Face de-identification framework · Human evaluation | Failure to identify the source image: 91.82% | Helped |
Surprising findings
- The learned-optics variant gives lower DIS than the fixed low-resolution camera baseline (Ours-LR) on most metrics, showing a privacy-utility trade-off in favor of the fixed low-resolution setup.
- Face-detection-based bounding box and landmark metrics show the method does not outperform RiDDLE, attributed to loss of high-frequency information.
Limitations
- Prototype deformable mirror supports only 15 Zernike coefficients, limiting the achievable scene distortion.
- Generated faces from privacy-preserving images still show artifacts and lower realism than RiDDLE.
- Face-detection and landmark metrics lag some SOTA methods such as RiDDLE due to high-frequency information loss.
Experimental setup
Datasets: CelebA-HQ FFHQ CASIA-Webface VGGFace2
Benchmarks: CelebA-HQ FFHQ
Metrics: DIS (l2 embedding distance) FID landmark distance bounding box distance face detection rate human evaluation protection rating PSNR under deconvolution
Baselines: CIAGAN FIT DeepPrivacy IDeudemon RiDDLE Ours-LR
Builds on: StarGAN2 StarGAN2 style encoder Zernike phase mask optics Adaptive Wing heatmap regression DDRM
Closest papers by meaning
- Disguise without Disruption: Utility-Preserving Face De-identification (AAAI 2024 · similarity 0.88)
- How Private is Private? A Comparative Study for Face De-Identification (NeurIPS 2026 · similarity 0.87)
- Validating Privacy-Preserving Face Recognition under a Minimum Assumption (CVPR 2024 · similarity 0.86)
- Privacy-Preserving Face Recognition Using Trainable Feature Subtraction (CVPR 2024 · similarity 0.86)
- Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation (ICML 2024 · similarity 0.84)
- Enhancing Facial Privacy Protection via Weakening Diffusion Purification (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.