Research map / Label-efficient and robust learning / Test-time adaptation
Stationary Latent Weight Inference for Unreliable Observations from Online Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SLWI, a Bayesian-filtering latent-weight framework that tracks unsupervised target weights alongside source weights in online test-time adaptation, with a non-stationarity score that backtracks unreliable weight updates; plugs into TENT, EATA, and ROID.
Problem
Online test-time adaptation (OTTA) under distribution shifts suffers catastrophic forgetting and error propagation from unreliable unsupervised target weights, and existing methods over-depend on source-model information and fail to accumulate target-domain knowledge.
Approach
Maintain a latent weight updated by a linear-Gaussian Kalman-style Bayesian filter that fuses the source weight and noisy target weights, and use a stationarity score on the posterior predictive to attenuate (backtrack) nonlinear, unreliable weight updates; the filtered latent weight replaces the source regularizer in the next OTTA step via a plug-in approximation.
Key results
- On ImageNet-C covariate shift (D2V), SLWI reaches 42.5% average error vs. 44.8% for ROID, the best prior method.
- On ImageNet-C label shift with gamma=0.0, SLWI reduces error from 17.4% (ROID) to 13.6%.
- Adding SLWI to ROID adds only about 2.4% GPU time (257.5s to 263.8s) while improving ImageNet-C covariate error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply SLWI instead of ROID | data2vec ViT-B · ImageNet-C covariate shifts | average error rate (%): 44.8±0.04 -> 42.5±0.03 | Helped |
| apply SLWI instead of ROID | data2vec ViT-B · D109 covariate shifts | average error rate (%): 44.2±0.06 -> 41.8±0.05 | Helped |
| apply SLWI instead of ROID | data2vec ViT-B · ImageNet-C label shifts, γ = 0.0 | average error rate (%): 17.4±0.21 -> 13.6±0.17 | Helped |
| apply SLWI instead of ROID | data2vec ViT-B · ImageNet-C label shifts, γ = 0.1 | average error rate (%): 41.3±0.03 -> 37.5±0.02 | Helped |
| apply SLWI instead of ROID | data2vec ViT-B · D109 label shifts, γ = 0.0 | average error rate (%): 29.3±0.03 -> 27.3±0.07 | Helped |
| apply SLWI instead of ROID | data2vec ViT-B · D109 label shifts, γ = 0.1 | average error rate (%): 36.3±0.07 -> 33.5±0.09 | Helped |
| add SLWI to TENT (ablation) | TENT on ImageNet-C · ImageNet-C covariate shifts | average error rate (%): 53.3±0.22 -> 51.1±0.07 | Helped |
| add SLWI to TENT (ablation) | TENT on ImageNet-C · ImageNet-C label shifts, γ = 0.0 | average error rate (%): 56.2±0.98 -> 27.0±0.07 | Helped |
| add SLWI to TENT (ablation) | TENT on ImageNet-C · ImageNet-C label shifts, γ = 0.1 | average error rate (%): 55.4±1.58 -> 50.8±0.15 | Helped |
| add SLWI to EATA (ablation) | EATA on ImageNet-C · ImageNet-C covariate shifts | average error rate (%): 51.8±0.08 -> 47.4±0.09 | Helped |
| add SLWI to EATA (ablation) | EATA on ImageNet-C · ImageNet-C label shifts, γ = 0.0 | average error rate (%): 48.2±0.34 -> 20.2±0.12 | Helped |
| add SLWI to EATA (ablation) | EATA on ImageNet-C · ImageNet-C label shifts, γ = 0.1 | average error rate (%): 46.8±0.06 -> 43.9±0.01 | Helped |
Surprising findings
- Plain TENT under label shift (gamma=0.0) degrades to 56.2% error, worse than the source model at 51.8% on ImageNet-C in the covariate setting; prior methods such as LAME degrade sharply (70.7%).
Limitations
- Requires hyperparameters (stationary level zeta, backtracking degree alpha, and Bayesian filter parameters a, q, b) for backtracking; authors note these are fixed rather than learned.
- Evaluated only on ImageNet-C and D109 with ViT-family backbones (D2V, ViT, Swin).
Experimental setup
Datasets: ImageNet-C D109 (DomainNet-based)
Benchmarks: ImageNet-C D109
Metrics: average error rate (%) standard deviation over seeds 0-4 GPU time
Baselines: TENT LAME RoTTA SAR EATA ROID
Builds on: TENT EATA ROID SAR RoTTA
Closest papers by meaning
- Bayesian Weight Enhancement with Steady-State Adaptation for Test-time Adaptation in Dynamic Environments (ICML 2025 · similarity 0.90)
- PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation (NeurIPS 2025 · similarity 0.87)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.86)
- ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains (NeurIPS 2025 · similarity 0.86)
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.86)
- Online Learning via Learned Latent Bayesian Tracking (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.