Research map / Label-efficient and robust learning / Domain adaptation and generalization

Reducing Divergence in Batch Normalization for Domain Adaptation

AAAI 2025 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Feature alignment for unsupervised adaptation · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Refined Batch Normalization (RBN), which replaces deeper BN layers with Group Normalization inside residual bottleneck blocks (RBNBlock) to reduce accumulated BN estimation shift for unsupervised domain adaptation, plugged into CDAN and other UDA baselines.

Problem

Batch normalization conflates source and target statistics and its population-statistic estimation shift accumulates through stacked BN layers, degrading target-domain performance in unsupervised domain adaptation.

Approach

Quantifies BN estimation shift (ESM) and mitigates its accumulation by swapping deeper BN layers for batch-free Group Normalization within a bottleneck block (RBNBlock), yielding a plug-and-play normalization module for UDA.

Key results

• CDAN+RBN improves average accuracy over CDAN by 2.5% on Office-31 (90.2 avg), 3.2% on ImageCLEF-DA (90.9 avg), and 4.4% on Office-Home (70.2 avg).
• On VisDA-2017, CDAN+RBN reports an 8.6% improvement over the CDAN baseline (82.6).
• RBN as a plug-in adds 1.1% to DANN, 0.9% to Source Only, and 1.1% to BSP on Office-31.

Measured effects reported in the paper

Surprising findings

• Larger batch size improves UDA performance, and estimation shift magnitude increases with depth of BN layers.

Limitations

• Gains are reported against a single main baseline (CDAN) and mostly on standard benchmarks; generalization to large-scale models is untested.
• The theoretical link between estimation shift and BN accumulation is argued largely empirically and is loosely formulated.

Experimental setup

Datasets: Office-31 ImageCLEF-DA Office-Home VisDA-2017

Benchmarks: Office-31 ImageCLEF-DA Office-Home VisDA-2017

Metrics: classification accuracy (%)

Baselines: CDAN AdaBN AutoDIAL TransNorm DANN BSP SHOT MCD

Builds on: Batch-Free Normalization CDAN AdaBN AutoDIAL Group Normalization

Closest papers by meaning

• Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019 · similarity 0.92)
• Cross-Domain Collaborative Normalization via Structural Knowledge (AAAI 2022 · similarity 0.92)
• Understanding Batch Normalization (NeurIPS 2018 · similarity 0.91)
• Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.89)
• Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability (AAAI 2024 · similarity 0.89)
• Leveraging Normalization Layer in Adapters with Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning (AAAI 2024 · similarity 0.88)