Research map / Federated learning and privacy / Personalized and heterogeneous FL

FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models

CVPR 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated learning with pretrained models · 14 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedBiP, which personalizes a pretrained Stable Diffusion LDM at instance level (client latents with noise) and concept level (learned domain/category textual concept vectors) to synthesize client-aligned training data for heterogeneous one-shot federated learning.

Problem

One-shot federated learning with feature-space and label-space heterogeneity and scarce client data; directly prompting pretrained LDMs yields synthetic images with large distribution shifts from client data, especially in rare domains like medical imaging.

Approach

Instead of uploading data or model weights, each client uploads privacy-perturbed noised latents (interpolated same-class latents plus T-step noise) and optimized domain/category concept embeddings; the server generates client-aligned synthetic images with the frozen pretrained LDM and trains a classifier on them.

Key results

• FedBiP-L averages 67.82% on DomainNet versus 61.49% for FedAvg (multi-round) and 73.42% for centralized oracle; on OfficeHome FedBiP-L reaches 61.52% vs 60.00% FedAvg.
• On DermaMNIST and UCM, FedBiP-L improves over FedAvg by up to 4.16% on average across label-heterogeneity splits.
• MIA entropy gap on DomainNet drops from 0.1311 (FedAvg) to 0.0186 with FedBiP (85.8% reduction).

Measured effects reported in the paper

Surprising findings

• Directly prompting the pretrained LDM with domain/category text sometimes yields worse accuracy than baseline methods.
• Optimizing only the category concept vector gives minimal gains, since general object categories are already well captured by pretraining.

Limitations

• Performance underperforms on domains overlapping with LDM pretraining data (e.g., DomainNet Real), where adaptation reduces generation diversity.
• Privacy analysis is largely qualitative plus a single membership inference attack; latent upload privacy is not formally guaranteed.
• Relies on a pretrained LDM whose pretraining domain coverage limits quality for rare domains.

Experimental setup

Datasets: DomainNet PACS OfficeHome DermaMNIST UC Merced Land Use Dataset

Benchmarks: DomainNet PACS OfficeHome DermaMNIST UC Merced Land Use Dataset

Metrics: classification accuracy membership inference attack entropy and loss gap

Baselines: FedAvg Central (oracle) FedD3 DENSE FedDEO FGL

Builds on: Latent Diffusion Model textual inversion FGL FedDEO DENSE

Closest papers by meaning

• One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models (ICML 2025 · similarity 0.90)
• MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis (ICML 2024 · similarity 0.87)
• FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.87)
• pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.87)
• Exploring One-Shot Semi-supervised Federated Learning with Pre-trained Diffusion Models (AAAI 2024 · similarity 0.87)
• Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.87)