Research map / Federated learning and privacy / Personalized and heterogeneous FL
Generalized and Personalized Federated Learning with Black-Box Foundation Models via Orthogonal Transformations
Eun Gyung Kong, Jewon Yeom, Yonghoon Jeon, Taesup Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add orthogonality constraint on local transform (vs FedLT) (ablation) | FedOT · FEMNIST | average generalization accuracy (%): 88.74 -> 94.93 | Improved |
| add orthogonality constraint on local transform (vs FedLT) (ablation) | FedOT · VLCS | pseudo-gradient cosine similarity: -0.084 -> 0.002 | Improved |
| use orthogonal linear instead of nonlinear MLP adapter (ablation) | FedOT · VLCS | pseudo-gradient cosine similarity: -0.307 -> 0.002 | Improved |
| add orthogonality constraint on local transform (vs FedLT) (ablation) | FedOT · 5-dataset average | comprehensive accuracy (C, %): 84.24 -> 83.65 | Hurt |
| add orthogonality constraint on local transform (vs FedLT) (ablation) | FedOT · TerraIncognita | average generalization accuracy (%): 29.87 -> 29.58 | No clear effect |
| add block-diagonal orthogonal transform | FedOT · 5-dataset average | comprehensive accuracy (C, %): 83.65 -> 86.10 | Improved |
| add client-specific local orthogonal transforms to all-global model (ablation) | FedOT · 5-dataset average | personalization accuracy (P, %): 84.09 -> 86.21 | Improved |
| add global classifier sharing to purely local training (ablation) | FedOT(+B) · 5-dataset average | personalization accuracy (P, %): 84.54 -> 88.58 | Improved |
| random classifier init instead of text-encoder init (ablation) | FedOT · FEMNIST | comprehensive accuracy (C, %): 95.84 -> 96.51 | Improved |
| random classifier init instead of text-encoder init (ablation) | FedOT · VLCS | comprehensive accuracy (C, %): 77.13 -> 74.23 | Hurt |
| scale participating clients from 1 to 75 | FedOT · FEMNIST cross-device | generalization accuracy (%): 44.00 -> 71.18 | Improved |
| increase participating clients from 40 to 75 | PromptFL · FEMNIST cross-device | generalization accuracy (%): 73.54 -> 65.30 | Hurt |
Closest papers by meaning
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.90)
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.90)
- FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity (ICLR 2024 · similarity 0.89)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.88)
- Overcoming Data and Model heterogeneities in Decentralized Federated Learning via Synthetic Anchors (ICML 2024 · similarity 0.88)
- Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
