Research map / Federated learning and privacy / Personalized and heterogeneous FL
FORLA: Federated Object-centric Representation Learning with Slot Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FORLA, a federated slot-attention framework that trains a shared feature adapter over frozen foundation-model features (DINO, SAM, MAE, CLIP) with a two-branch student-teacher design and local FedAvg, for unsupervised object discovery across surgical and natural datasets.
Problem
Learning disentangled, object-centric visual representations from heterogeneous, unlabeled, non-IID datasets distributed across clients without sharing data.
Approach
A shared lightweight adapter and shared slot attention module are trained with FedAvg across clients; a two-branch design has a student decoder reconstructing raw concatenated foundation features and a teacher decoder reconstructing adapted features, with teacher and student slot modules synchronized by local FedAvg instead of a knowledge-distillation loss, avoiding slot permutation-matching issue
Key results
- FORLA with AFM adapter reaches 80.3 CorLoc on Abdominal and 75.42 on Thoracic in the stratified setting, outperforming individual and centralized training.
- The paper claims 7.6% higher mIoU in multi-domain object discovery and 85.2% communication reduction versus fine-tuning foundation models, and up to 6.7x lower communication cost than mixture-of-foundation-models.
- On mixed training of all seven datasets, FORLA + AFM reaches 63.02 CorLoc average-style results (average mBO 40.33 for Individual+AFM vs 41.45 Centralized+AFM; FORLA+AFM average mBO 44.63 in the Surgery|Natural row).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use federated FORLA + AFM instead of centralized AFM training | Object discovery on mixed-domain data · Mixed-domain Average | mBO: 41.45 -> 43.81 | Helped |
| Use federated FORLA + AFM instead of individual AFM training | Object discovery on mixed-domain data · Mixed-domain Average | CorLoc: 56.82 -> 63.02 | Helped |
| Use the AFM adapter instead of MLP or MoE in FORLA (ablation) | FORLA · Mixed-domain Average | mBO: 41.53; 42.29; 43.81 | Helped |
| Replace FedAvg with FedProx in FORLA + MoE (ablation) | FORLA + MoE · Mixed-domain Average | CorLoc: 60.37 -> 59.89 | Hurt |
| Replace FedAvg with FedAdam in FORLA + MoE (ablation) | FORLA + MoE · Mixed-domain Average | CorLoc: 60.37 -> 59.01 | Hurt |
| Add teacher-student self-distillation and two-stage reconstruction (ablation) | Federated Slot FedAvg · Mixed-domain Average | mBO: 41.92 -> 43.81 | Helped |
| Use two-stage instead of one-stage distillation (ablation) | Individual training · Abdominal | mBO: 33.91 -> 56.93 | Helped |
| Use two-stage instead of one-stage distillation (ablation) | Individual training · Abdominal | CorLoc: 29.22 -> 79.40 | Helped |
| Add two-stage distillation to individual training (ablation) | Individual training · Cholec | mBO: 35.35 -> 32.50 | Hurt |
| Add two-stage distillation to individual training (ablation) | Individual training · YTOBJ | CorLoc: 65.39 -> 63.97 | Hurt |
| Personalize Slot Attention after global FORLA training (ablation) | FORLA · Abdominal | mBO: 57.04 -> 60.21 | Helped |
| Personalize the adapter after global FORLA training (ablation) | FORLA · YTVIS | mBO: 37.72 -> 38.79 | Helped |
Surprising findings
- Centralized training on mixed natural and surgical data can underperform individual training for some subdomains (e.g., YTOBJ mBO drops from 48.98 individual to 44.51 centralized with AFM).
- Slot FedAvg performs well on natural data with MoE adapter but struggles on surgical data, suggesting teacher supervision matters most for domains underrepresented in foundation-model pretraining.
Limitations
- Relies on frozen foundation models and struggles to discover objects purely from image reconstruction.
- Assumes an optimal decomposition hierarchy and a fixed slot count per dataset; hierarchical compositionality is not modeled.
- Federated advantage declines as per-client data becomes scarce (10% splits), and centralized mixed training shows dilution effects.
Experimental setup
Datasets: Abdominal surgical dataset Cholec80 Thoracic surgery dataset (proprietary, open-access version released) COCO PASCAL VOC 2012 YouTube-VIS YouTube-Objects
Benchmarks: Abdominal Cholec80 Thoracic COCO PASCAL VOC 2012 YouTube-VIS YouTube-Objects
Metrics: mBO FG-ARI CorLoc mBHD
Baselines: Individual training Centralized mixed training Concat features Slot FedAvg FT-DINOSAUR SAM ViT-H zero-shot segmentation
Builds on: Slot Attention DINOSAUR FedAvg DINO
Closest papers by meaning
- Adaptive Slot Attention: Object Discovery with Dynamic Slot Number (CVPR 2024 · similarity 0.86)
- FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction (NeurIPS 2024 · similarity 0.86)
- Enhancing Foundation Models with Federated Domain Knowledge Infusion (ICML 2025 · similarity 0.86)
- MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning (NeurIPS 2025 · similarity 0.85)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.85)
- Global and Local Prompts Cooperation via Optimal Transport for Federated Learning (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.