Research map / Architectures and efficiency / RNNs and normalization
Dynamic Group Normalization: Spatio-Temporal Adaptation to Evolving Data Statistics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dynamic Group Normalization (DGN), which periodically regroups channels by intra-group statistical similarity (KMeans with size bounds) during training, and reports gains over GN, LN and GNP across classification, detection, segmentation, long-tail, OOD and corruption benchmarks.
Problem
Fixed-size normalization groups (e.g., Group Normalization) mix channels with dissimilar statistics, which hurts adaptability to evolving data statistics, minority classes, OOD inputs and corrupted data.
Approach
Instead of fixed channel groups, DGN periodically reassigns channels into variable-size groups using constrained KMeans on channel statistics (mean and variance), reassigning only the most heterogeneous groups selected by a percentile variance threshold, on a fixed epoch schedule.
Key results
- DGN averages +0.98% accuracy over GN, +1.45% over LN and +2.55% over GNP in image classification across models and datasets.
- On YOLOv9-C COCO, DGN reaches 53.89 mAP@50:95 vs 51.88 for GN.
- On CIFAR-100-LT (IF=100), DGN averages 1.28% gains over GN; on OOD (ImageNet to ImageNet-O) DGN raises AUROC for MobileNetV3-small from 70.74 to 75.51.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GN with DGN | ResNet50 · CIFAR-100 | accuracy: 72.48_{0.61} -> 73.44_{0.69} | Helped |
| replace GN with DGN | YOLOv9-C · COCO | mAP@50: 68.40_{0.53} -> 70.73_{0.48} | Helped |
| replace GN with DGN | DeepLabV3+HRNetV2-48 · Cityscapes | Mean IoU: 57.17_{0.83} -> 63.06_{0.66} | Helped |
| replace GN with DGN | ViT-Base (LIFT) · ImageNet-LT | accuracy: 73.67_{0.32} -> 76.33_{0.26} | Helped |
| replace GN with DGN | MobileNetV3-small · ImageNet → ImageNet-O | AUROC: 70.74_{0.38} -> 75.51_{0.36} | Helped |
| replace GN with DGN | YOLOv5-X · VOC → COCO | FPR95: 63.87_{0.45} -> 61.02_{0.34} | Helped |
| replace GN with DGN | ResNext-29A · CIFAR-100-C | CER: 26.33 -> 25.47 | Helped |
| add partial reassignment (p > 0) (ablation) | DGN baseline · CIFAR-100 | accuracy: 70.34_{0.63} -> 71.02_{0.64} | Helped |
| add variable group sizes (α > 0) (ablation) | DGN baseline · CIFAR-100 | accuracy: 70.34_{0.63} -> 71.52_{0.64} | Helped |
| add variable group sizes and partial reassignment (ablation) | DGN baseline · CIFAR-100 | accuracy: 70.34_{0.63} -> 72.21_{0.71} | Helped |
| use the proposed scheduling mechanism rather than naive every-epoch regrouping (ablation) | DGN · training | accuracy: +3.42% | Helped |
| regroup every epoch within [Einit, Efinal] (ablation) | DGN · training | accuracy | Hurt |
Surprising findings
- Reassigning every epoch within the regrouping window degraded performance due to destabilized feature learning, so the scheduled interval matters.
Limitations
- Relies on a hand-tuned schedule (E_init, E_interval, E_final) and thresholds (p, alpha) selected by grid search.
- Most models trained from scratch; gains on pre-trained models are described as less systematically evaluated.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet MS-COCO Cityscapes CIFAR100-LT ImageNet-LT ImageNet-O CIFAR-100-C ImageNet-C ImageNet-S PASCAL VOC
Benchmarks: CIFAR-10 CIFAR-100 ImageNet MS-COCO Cityscapes CIFAR100-LT ImageNet-LT ImageNet-O CIFAR-100-C ImageNet-C ImageNet-S
Metrics: accuracy mAP@50 mAP@50:95 mean IoU AUROC FPR95 CER mCE
Baselines: GN LN GNP LCN
Builds on: Group Normalization Batch Normalization Local Context Normalization GNP
Closest papers by meaning
- Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning (NeurIPS 2021 · similarity 0.87)
- Group Whitening: Balancing Learning Efficiency and Representational Capacity (CVPR 2021 · similarity 0.86)
- Kalman Normalization: Normalizing Internal Representations Across Network Layers (NeurIPS 2018 · similarity 0.85)
- Flexible Group Count Enables Hassle-Free Structured Pruning (CVPR 2025 · similarity 0.83)
- Unsupervised Group Re-identification via Adaptive Clustering-Driven Progressive Learning (AAAI 2024 · similarity 0.83)
- When Dynamic Data Selection Meets Data Augmentation: Achieving Enhanced Training Acceleration (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.