Research map / Probabilistic and scientific modeling / Molecules and proteins
Reimagining Mutual Information for Enhanced Defense against Data Leakage in Collaborative Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InfoScissors, an adversarial mutual-information (CLUB-based) regularization defense that limits input and label leakage in edge-cloud collaborative inference while preserving accuracy.
Problem
Edge-cloud collaborative inference leaks input data (via model inversion from intermediate representations) and predictions (via model completion attacks on server-side features), and existing defenses degrade accuracy significantly, especially when the on-device head model is shallow.
Approach
Bound mutual information I(r;x) and I(z;y) with a CLUB-based variational upper bound and train the device head adversarially against an auxiliary inversion generator (and the server encoder against a label classifier), rather than using VIB-style Gaussian regularization of representations.
Key results
- On CIFAR10, InfoScissors reduces KA reconstruction SSIM below 0.2 with under 2% accuracy drop, whereas baselines lose more than 10% accuracy.
- Against PMC on CIFAR10, InfoScissors reaches ~10% attack accuracy (random guess) with under 0.5% accuracy sacrifice, while baselines lose more than 4%.
- Integrated input and prediction protection keeps accuracy drop under 2% on both CIFAR10 and CIFAR100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply InfoScissors input protection | Collaborative inference with ResNet18 · CIFAR10, KA attack | reconstruction SSIM: SSIM ... lower than 0.2; model accuracy drop of less than 2% | Helped |
| Apply InfoScissors input protection | Collaborative inference with ResNet18 · CIFAR100, KA attack | reconstruction SSIM: SSIM of lower than 0.2; model accuracy drop of less than 2% | Helped |
| Apply InfoScissors input protection | Collaborative inference with ResNet18 · CIFAR10 and CIFAR100, rMLE attack | reconstruction SSIM: SSIM to lower than 0.2 with a model accuracy drop of less than 2% for CIFAR10 and 1% for CIFAR100 | Helped |
| Apply InfoScissors prediction protection | Collaborative inference with ResNet18 · CIFAR10, PMC attack | attack accuracy: 10% attack accuracy ... sacrificing less than 0.5% model accuracy | Helped |
| Apply InfoScissors prediction protection | Collaborative inference with ResNet18 · CIFAR100, PMC attack | attack accuracy: 1% attack accuracy ... sacrificing less than 1% model accuracy | Helped |
| Apply InfoScissors prediction protection | Collaborative inference with ResNet18 · CIFAR10, AMC attack | attack accuracy: less than 1% ... model accuracy on CIFAR10 | Helped |
| Apply InfoScissors prediction protection | Collaborative inference with ResNet18 · CIFAR100, AMC attack | attack accuracy: less than 0.5% ... model accuracy on CIFAR100 | Helped |
| Integrate input and prediction protection with InfoScissors | Collaborative inference with ResNet18 · CIFAR10 and CIFAR100, KA and PMC attacks | model accuracy: less than a 2% accuracy drop | Helped |
| Fine-tune a classifier using the collaborative encoder rather than train from scratch | Server-side classifier on CIFAR10 · CIFAR10, 40 labels | accuracy (%): 15.34 -> 69.31 | Helped |
Surprising findings
- Prior VIB-based defenses (e.g., MID) only work well when the device head is deep, and the paper attributes this to VIB forcing representations toward a fixed Gaussian, discarding label-relevant information.
Limitations
- Only a single edge device is considered; extension to multiple devices is not evaluated.
- Evaluated only on CIFAR10 and CIFAR100 with ResNet18 backbone.
- Theoretical guarantees depend on the variational approximations and an assumed Gaussian p(x|r) with unit variance.
Experimental setup
Datasets: CIFAR10 CIFAR100
Benchmarks: CIFAR10 CIFAR100
Metrics: test accuracy SSIM attack accuracy
Baselines: Differential Privacy Adding Noise Data Compression PPDL MID
Builds on: CLUB Variational Information Bottleneck MID Knowledge Alignment attack
Closest papers by meaning
- Occult: Optimizing Collaborative Communications across Experts for Accelerated Parallel MoE Training and Inference (ICML 2025 · similarity 0.84)
- Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks (AAAI 2024 · similarity 0.83)
- Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models (ICML 2025 · similarity 0.82)
- HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference (ICML 2026 · similarity 0.82)
- Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.82)
- Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training (ICML 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.