Research map / Federated learning and privacy / Personalized and heterogeneous FL

Multifaceted User Modeling in Recommendation: A Federated Foundation Models Approach

AAAI 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MRFF, a lightweight transformer recommender trained from scratch on each federated client, with a group gating network routing users to group-level FFNs plus private user-specific FFNs, and a balance loss for even group assignment.

Problem

Foundation-model recommenders need large pre-trained models and aggregated user data, which is costly on client devices and raises privacy risks; naive federated foundation models also treat all users identically rather than personalizing.

Approach

Insert a group gating network after attention in each transformer block that routes users to shared group-level FFNs, alongside a private user-specific FFN; the server aggregates shared parameters globally or by group while user embeddings and user FFNs stay local, with a balance loss keeping groups uniform.

Key results

• On KuaiRand-Pure, FedHSTU w/ MRFF reaches AUC 0.7330 vs 0.7304 for FedHSTU, and LogLoss 0.4949 vs 0.5171.
• On KuaiSAR-S, FedSASRec w/ MRFF lowers LogLoss from 0.7034 to 0.5259 (33.75% improvement reported).
• Each client model has roughly 60k parameters with average testing time under 0.1 seconds across SASRec, HSTU and LLaMA backbones.

Measured effects reported in the paper

Surprising findings

• Adding differential-privacy noise up to strength 0.5 leaves AUC nearly unchanged on KuaiRand-Pure and KuaiSAR-R, while LogLoss on KuaiSAR-S rises substantially under noise.

Limitations

• Gains on some metrics are small (e.g., AUC improvements of 0.12% to 1.16%) and differential privacy degrades LogLoss on KuaiSAR-S.
• Balance loss is one possible constraint; alternatives were not compared in depth.
• Evaluated only on CTR prediction with three datasets and 500 communication rounds.

Experimental setup

Datasets: KuaiRand-Pure KuaiSAR-R KuaiSAR-S

Benchmarks: KuaiRand-Pure KuaiSAR-R KuaiSAR-S

Metrics: AUC LogLoss

Baselines: FedNCF FedPA FedSASRec FedHSTU FedLLaMA

Builds on: SASRec HSTU LLaMA FedNCF FedPA

Closest papers by meaning

• Beyond Single Embedding: Modeling User Preferences as Distribution in Federated Recommendation (ICML 2026 · similarity 0.89)
• Federated Context-Aware Personalized Recommendation (AAAI 2026 · similarity 0.88)
• Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach (AAAI 2025 · similarity 0.87)
• Federated Recommendation with Additive Personalization (ICLR 2024 · similarity 0.86)
• FACE: A General Framework for Mapping Collaborative Filtering Embeddings into LLM Tokens (NeurIPS 2025 · similarity 0.86)
• LIME: Link-based User-item Interaction Modeling with Decoupled XOR Attention for Efficient Test Time Scaling (NeurIPS 2026 · similarity 0.85)