Research map / Label-efficient and robust learning / Test-time adaptation
Test-time Adaptation for Regression by Subspace Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Significant-subspace Alignment (SSA), a test-time adaptation method for regression that aligns target features to source statistics within a PCA-detected significant feature subspace, weighted by regressor importance.
Problem
Test-time adaptation for regression under covariate shift, where classification-specific TTA methods (entropy minimization, naive feature alignment) are inapplicable or harmful because regression features occupy a small subspace.
Approach
Detect the low-dimensional PCA subspace of source features and perform KL-based feature alignment only within it, with per-dimension weights proportional to 1 + |w^T v_d| reflecting each basis direction's effect on the scalar output; only normalization affine parameters are updated.
Key results
- On SVHN-MNIST, SSA reaches R^2 0.511 versus 0.406 for the source model and 0.491 for the best baseline (Prototype).
- On UTKFace with 13 corruptions, SSA attains mean R^2 0.731 versus 0.700 for FR and -0.678 for the source model.
- On Biwi Kinect, SSA averages R^2 0.778 versus 0.706 for the source; naive feature alignment without subspace detection underperforms the source on SVHN and Biwi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add significant-subspace alignment | ResNet-26 regression model · SVHN-MNIST | R²: 0.406 -> 0.511 | Helped |
| add significant-subspace alignment | MLP regression model · California Housing | R²: 0.605 -> 0.639 | Helped |
| add significant-subspace alignment | ResNet-50 regression model · UTKFace corruption mean | R²: -0.678 -> 0.731 | Helped |
| add significant-subspace alignment | ResNet-50 regression model · Biwi Kinect mean | R²: 0.706 -> 0.778 | Helped |
| add subspace detection and align the representative feature subspace (ablation) | SSA · SVHN-MNIST | R²: 0.338 -> 0.511 | Helped |
| add subspace detection and align the representative feature subspace (ablation) | SSA · UTKFace corruption mean | R²: 0.641 -> 0.731 | Helped |
| add subspace detection and align the representative feature subspace (ablation) | SSA · Biwi Kinect mean | R²: 0.672 -> 0.778 | Helped |
| add subspace detection and align the representative feature subspace (ablation) | SSA · California Housing | R²: 0.626 -> 0.639 | Helped |
| add dimension weighting (ablation) | SSA · SVHN-MNIST | R²: 0.508 -> 0.511 | Helped |
| add dimension weighting (ablation) | SSA · UTKFace corruption mean | R²: 0.728 -> 0.731 | Helped |
| add dimension weighting (ablation) | SSA · Biwi Kinect mean | R²: 0.778 -> 0.778 | No clear effect |
| add dimension weighting (ablation) | SSA · California Housing | R²: 0.633 -> 0.639 | Helped |
Surprising findings
- Naive feature alignment over all feature dimensions is ineffective or even worse than the source model for regression, often diverging due to zero-variance dimensions.
- Features of regression models lie in a small subspace (fewer than 100 dimensions of 2048 in ResNets), and ReLU causes many dimensions to have zero variance.
Limitations
- Assumes covariate shift, so p(y|x) must be unchanged; concept drift is not handled.
- Hyperparameter K must be chosen, and large K causes instability; K is guided by the source subspace dimension.
- Gaussian assumption for KL is heuristic, since features are not truly Gaussian under ReLU.
Experimental setup
Datasets: SVHN MNIST UTKFace Biwi Kinect California Housing ImageNet-C corruptions
Benchmarks: SVHN-MNIST UTKFace with corruptions Biwi Kinect California Housing
Metrics: R^2 RMSE MAE
Baselines: Source BN-adapt Feature restoration (FR) Prototype (T3A-based) Variance minimization (VM) RSD TTT DANN Oracle
Builds on: Tent T3A Feature restoration Test-time training Representation subspace distance
Closest papers by meaning
- Test-time Correlation Alignment (ICML 2025 · similarity 0.89)
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.88)
- Bayesian Weight Enhancement with Steady-State Adaptation for Test-time Adaptation in Dynamic Environments (ICML 2025 · similarity 0.87)
- TEA: Test-time Energy Adaptation (CVPR 2024 · similarity 0.87)
- Monitoring Risks in Test-Time Adaptation (NeurIPS 2025 · similarity 0.86)
- Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.