Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Bayesian Weight Enhancement with Steady-State Adaptation for Test-time Adaptation in Dynamic Environments

ICML 2025 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Nonparametric and PAC-Bayes Bayesian learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a Bayesian weight enhancement framework for test-time adaptation and a steady-state adaptation (SSA) algorithm that adds covariance-aware learning-rate adjustment to weight-averaging TTA methods like ROID and CMF.

Problem

Test-time adaptation in dynamic environments suffers from weight degradation caused by explicit gradient noise from unsupervised entropy-based objectives, and existing weight-based TTA methods ignore time-varying covariance.

Approach

Model TTA weight updates as an SDE approximation of SGD and apply Bayesian (Kalman-style) filtering with a source-weight prior, then choose a step-size scalar so the posterior covariance stays at steady state; this generalizes existing weight-averaging TTA methods (time-invariant covariance) and adds a covariance-aware learning-rate adjustment.

Key results

• On ImageNet-C covariate shift, CMF+SSA reduces average error from 43.5% to 42.2% and ROID+SSA from 44.4% to 43.9%.
• In the label shift scenario with gamma=0.0, CMF+SSA reaches 13.1% average error versus 14.4% for CMF alone.
• SSA adds only 1.0% to 4.3% relative runtime to TENT, ROID, and CMF, and TENT+SSA improves average error from 53.3% to 49.3%.

Measured effects reported in the paper

Surprising findings

• Plain TENT with SSA gains about 4 points, suggesting the covariance-aware step size helps even without weight averaging.
• Baselines such as LAME and EATA degrade badly in some label-shift or periodic settings relative to the source model, while SSA-enhanced weight methods stay stable.

Limitations

• The SDE approximation assumes a small learning rate; SSA performance declines beyond about 15x the default learning rate.
• Evaluation is limited to ImageNet-C and D109 with a single ViT/data2vec backbone.
• Covariance is restricted to a scalar multiple of identity, a strong simplification of the weight covariance.

Experimental setup

Datasets: ImageNet-C D109 ImageNet

Benchmarks: ImageNet-C D109

Metrics: average error rate (%) standard deviation over five seeds elapsed time

Baselines: Source TENT LAME RoTTA SAR EATA DeYO ROID CMF

Builds on: ROID CMF SAR SDE approximation of SGD (Li et al., 2019) Kalman filtering

Closest papers by meaning

• Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.90)
• Stationary Latent Weight Inference for Unreliable Observations from Online Test-Time Adaptation (ICML 2024 · similarity 0.90)
• A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.89)
• Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.88)
• Evaluation of Test-Time Adaptation Under Computational Time Constraints (ICML 2024 · similarity 0.88)
• TEA: Test-time Energy Adaptation (CVPR 2024 · similarity 0.88)