Research map / Federated learning and privacy / Personalized and heterogeneous FL
Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedHP, a federated learning framework that trains a hardware-conditioned prompter plus adaptors on a frozen pre-trained SCI reconstruction backbone to handle heterogeneous coded apertures across clients, and releases a Snapshot Spectral Heterogeneous Dataset (SSHD).
Problem
Snapshot compressive imaging reconstruction models trained on a single calibrated hardware instance degrade under hardware shifts, and centralized multi-hardware training violates data privacy and cannot handle heterogeneous coded apertures across institutions.
Approach
Instead of correcting client drift in gradient space as prior FL methods do, learn a hardware-conditioned prompt network that aligns input measurements across clients in the data space, while keeping the pre-trained reconstruction backbone frozen and training only prompts and lightweight adaptors that are federated.
Key results
- FedHP reaches 31.35 dB average PSNR vs 31.21 dB for FedAvg under non-overlapping masks (3 clients).
- Under distinct coded-aperture distributions, FedHP averages 28.98 dB vs 28.63 dB for FedAvg, while FedProx and SCAFFOLD drop to about 20-21 dB.
- FedHP uses 0.27M trainable parameters versus 0.12M for FedAvg's backbone-level training cost is reported as 10.62 vs 2.86 training days; abstract claims 0.35 dB gain under heterogeneous settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedHP instead of FedAvg | MST-S for CAVE/KAIST with non-overlapping masks · CAVE/KAIST, non-overlapping masks (Avg.) | PSNR: 31.21 -> 31.35 | Helped |
| use FedHP instead of FedAvg | MST-S for CAVE/KAIST with non-overlapping masks · CAVE/KAIST, non-overlapping masks (Avg.) | SSIM: 0.8959 -> 0.9033 | Helped |
| use FedHP instead of FedAvg | MST-S with distinct client mask distributions · SSHD/KAIST, distinct client mask distributions (Avg.) | PSNR: 28.63 -> 28.98 | Helped |
| use FedHP instead of FedAvg | MST-S with distinct client mask distributions · SSHD/KAIST, distinct client mask distributions (Avg.) | SSIM: 0.8496 -> 0.8481 | Mixed |
| add federated learning (ablation) | FedHP without FL · CAVE/KAIST, non-overlapping masks | PSNR: 30.75 -> 31.35 | Helped |
| add federated learning (ablation) | FedHP without FL · CAVE/KAIST, non-overlapping masks | SSIM: 0.8890 -> 0.9033 | Helped |
| add adaptor (ablation) | FedHP without adaptor · CAVE/KAIST, non-overlapping masks | PSNR: 31.09 -> 31.35 | Helped |
| add adaptor (ablation) | FedHP without adaptor · CAVE/KAIST, non-overlapping masks | SSIM: 0.8996 -> 0.9033 | Helped |
| add hardware prompter (ablation) | FedHP without prompter · CAVE/KAIST, non-overlapping masks | PSNR: 19.19 -> 31.35 | Helped |
| add hardware prompter (ablation) | FedHP without prompter · CAVE/KAIST, non-overlapping masks | SSIM: 0.2303 -> 0.9033 | Helped |
| use FedHP instead of FedAvg | SCI reconstruction on CACTI · CACTI | PSNR: 27.35 -> 27.87 | Helped |
| use FedHP instead of FedAvg | SCI reconstruction on CACTI · CACTI | SSIM: 0.9174 -> 0.9192 | Helped |
Surprising findings
- FedProx and SCAFFOLD collapse to around 20 dB under distinct coded-aperture distributions, well below FedAvg and the no-FL baseline.
- Removing the prompter while keeping FL and adaptors drops PSNR to 19.19 dB, showing the prompt is essential.
Limitations
- Only 3 clients in the main experiments because of limited real hardware; scalability to many clients (C>100) untested.
- Real hardware data is private and limited, so the number of real SCI systems is small.
- No theoretical convergence analysis of FedHP.
Experimental setup
Datasets: CAVE KAIST SSHD
Benchmarks: KAIST simulation test set real SD-CASSI captures
Metrics: PSNR SSIM
Baselines: FedAvg FedProx SCAFFOLD FedGST GAP-Net Jointly train Self-tuning
Builds on: FedAvg MST GST visual prompt tuning
Closest papers by meaning
- Effective Snapshot Compressive-Spectral Imaging via Deep Denoising and Total Variation Priors (CVPR 2021 · similarity 0.86)
- Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms (NeurIPS 2024 · similarity 0.84)
- SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image (CVPR 2024 · similarity 0.83)
- Dual Prior Unfolding for Snapshot Compressive Imaging (CVPR 2024 · similarity 0.82)
- DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral Diffusion Model (CVPR 2024 · similarity 0.82)
- FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.