Research map / Architectures and efficiency / RNNs and normalization
Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CEMA, a cloud-edge test-time adaptation paradigm where edge models only run forward passes, with entropy-based sample filtering for upload and replay-based KD from a ResNet101 foundation model to a ResNet18 edge model updating only BN affine parameters.
Problem
Edge devices with limited compute cannot run backpropagation-based test-time adaptation under distribution shift, and uploading all test samples to the cloud for adaptation creates heavy communication costs.
Approach
Offload all adaptation to the cloud: filter uploads with a dynamic high-entropy threshold and a low-entropy threshold, store uploads in a replay buffer, and distill a foundation model (updated by entropy minimization) into the edge model by updating only normalization-layer affine parameters, which are then sent back to the edge.
Key results
- On ImageNet-C severity 5, CEMA reaches 40.8% average accuracy versus 38.7% for ETA and 35.1% for Tent with a ResNet18 edge model.
- On ImageNet-R, CEMA achieves 27.9% accuracy (+4.1% over Tent, +1.7% over ETA) while uploading 5944 samples (20%) versus 7457 for ETA.
- CEMA reduces uploaded samples by about 60% relative to methods uploading all test data, and on ImageNet-C severity 3 it uploads 18.8k vs 26.1k for ETA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CEMA rather than ETA | ResNet18 edge model · ImageNet-C severity 3 | accuracy (%): 53.5 -> 54.2 | Helped |
| Apply CEMA rather than ETA | ResNet18 edge model · ImageNet-C severity 5 | accuracy (%): 38.7 -> 40.8 | Helped |
| Apply CEMA rather than ETA | ResNet18 edge model · ImageNet-R | accuracy (%): +1.7% | Helped |
| Add replay-buffer samples to distillation (ablation) | CEMA edge-model adaptation · ImageNet-C Gaussian noise severity 3 | accuracy (%): 47.4% -> 51.1% | Helped |
| Use dynamic rather than fixed E_max (ablation) | CEMA sample filtration · ImageNet-C severity 3 | accuracy (%): 51.2% -> 51.1% | No clear effect |
| Use dynamic rather than fixed E_max (ablation) | CEMA sample filtration · ImageNet-C severity 3 | uploaded samples: 25,325 -> 20,976 | Helped |
| Also exclude low-entropy samples (ablation) | CEMA sample filtration with dynamic E_max · ImageNet-C severity 3 | uploaded samples: 25,325 -> 17,479 | Helped |
| Increase E_min above 0.02 (ablation) | CEMA sample filtration · ImageNet-C and ImageNet-R | accuracy | Hurt |
| Increase lambda up to 1.0 (ablation) | CEMA dynamic entropy threshold · ImageNet-C Gaussian noise severity 3 | robust accuracy (%): 51.1% | Helped |
| Increase lambda beyond 1.0 (ablation) | CEMA dynamic entropy threshold · ImageNet-C Gaussian noise severity 3 | uploaded samples | Hurt |
Surprising findings
- Adaptation gains are smaller at severity level 1, where the foundation model performs close to the edge model, so knowledge transfer is limited.
- Entropy-based high-entropy filtering with a fixed threshold progressively removes fewer samples as adaptation proceeds, motivating the dynamic threshold.
Limitations
- Bandwidth variability is not handled in the main method; the proposed upload-queue mechanism is discussed but not evaluated.
- Experiments are limited to image classification on ImageNet-C and ImageNet-R with CNN edge models in the main results.
- Gains are small at low severity levels where the foundation model offers little extra knowledge.
Experimental setup
Datasets: ImageNet-C ImageNet-R
Benchmarks: ImageNet-C ImageNet-R
Metrics: accuracy (%) number of uploaded samples ECE
Baselines: BN Adaptation ONDA LAME Pseudo Label Tent CoTTA ETA
Builds on: Tent ETA EATA CoTTA
Closest papers by meaning
- Aligning Logits Generatively for Principled Black-Box Knowledge Distillation (CVPR 2024 · similarity 0.83)
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.83)
- ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections (ICML 2024 · similarity 0.83)
- CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation (CVPR 2025 · similarity 0.83)
- AdMiT: Adaptive Multi-Source Tuning in Dynamic Environments (CVPR 2025 · similarity 0.82)
- HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference (ICML 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.