Research map / Adversarial robustness and security / Attacks on text and graphs

Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts

NeurIPS 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Membership inference and inversion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SHAPOOL, which trains a Mixture-of-Experts shadow pool so that activated pathways act as shared shadow models, cutting shadow-model construction cost by about 7.7x on LiRA and 75% on RMIA while keeping comparable membership inference attack performance.

Problem

Shadow model-based inference attacks (e.g., membership inference) need many shadow models, and training them independently is computationally expensive.

Approach

Train a Mixture-of-Experts model whose end-to-end activated pathways serve as shared shadow models, using pathway-choice routing with disjoint data allocation, a pathway regularizer (KL similarity plus orthogonality of expert outputs), and fine-tuning alignment on a small subset to match independently trained models.

Key results

• Under unconstrained budget, SHAPOOL reduces LiRA-offline shadow construction cost by an average of 88.6% (about 7.7x) with comparable or higher AUC on CIFAR100 and CIFAR10 (e.g., ResNet18 CIFAR100 AUC 0.84 vs 0.81).
• On RMIA with CIFAR100 and ResNet18, SHAPOOL cuts cost by 76% (offline) and 75% (online) with AUC of 0.93 vs 0.94 for the baseline.
• Under constrained budget, SHAPOOL improves LiRA-offline AUC on CIFAR100 with ResNet18 from 0.73 to 0.82 at similar cost.

Measured effects reported in the paper

Surprising findings

• Naive model augmentation via neural masking of independently trained shadow models fails to improve attacks, with diversity and consistency suffering.
• Distilled shadow models (R2) perform notably worse than the baseline in LiRA-online (AUC 0.85 vs 0.94 at 64 models).

Limitations

• Requires modifying the target architecture into an MoE form, which entails network architecture changes.
• Performance may degrade in low-data attack scenarios because MoE has more parameters needing more auxiliary data.
• LiRA-online TPR at low FPR slightly reduced in the unconstrained setting, attributed to sensitivity of online attacks to shadow behavior.

Experimental setup

Datasets: CIFAR100 CIFAR10 CINIC10 TinyImageNet Census

Benchmarks: CIFAR100 CIFAR10 CINIC10 TinyImageNet Census

Metrics: AUC TPR@FPR=1% TPR@FPR=0.1% wall-clock cost (hours) accuracy (PIA)

Baselines: BASE (independent shadow models) LiRA RMIA Distilled (R2) model augmentation (neural masking)

Builds on: Mixture-of-Experts LiRA RMIA neural masking

Closest papers by meaning

• Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast (NeurIPS 2024 · similarity 0.85)
• Occult: Optimizing Collaborative Communications across Experts for Accelerated Parallel MoE Training and Inference (ICML 2025 · similarity 0.84)
• A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts (ICML 2024 · similarity 0.83)
• Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations (NeurIPS 2025 · similarity 0.83)
• Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning (ICLR 2024 · similarity 0.83)
• Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers (NeurIPS 2025 · similarity 0.83)