Research map / Architectures and efficiency / RNNs and normalization
REPAIR: REnormalizing Permuted Activations for Interpolation Repair
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add REPAIR channel-statistics correction to aligned ResNet18 interpolation | ResNet18 · CIFAR-10 test, midpoint | accuracy: 66.0% -> 94.1% | Helped |
| Add REPAIR channel-statistics correction | BatchNorm ResNet20, 8× · CIFAR-10, midpoint | accuracy: 32.3% -> 94.4% | Helped |
| Add REPAIR channel-statistics correction | LayerNorm ResNet20, 8× · CIFAR-10, midpoint | accuracy: 90.3% -> 93.2% | Helped |
| Add REPAIR to aligned ResNet18 interpolation | ResNet18 · ImageNet validation, midpoint | accuracy: below 1% -> 41.1% | Helped |
| Add REPAIR to aligned ResNet50 interpolation | ResNet50 · ImageNet validation, midpoint | accuracy: below 1% -> 56.5% | Helped |
| Add REPAIR to aligned double-width ResNet50 interpolation | Double-width ResNet50 · ImageNet validation, midpoint | accuracy: below 1% -> 64.2% | Helped |
| Also correct residual-block outputs, in addition to convolutional outputs (ablation) | REPAIR on ResNet50 · ImageNet validation, midpoint | accuracy: 53.2% -> 56.5% | Helped |
| Use LayerNorm instead of BatchNorm (ablation) | ResNet20 · CIFAR-10 test, single endpoint | accuracy: 93.6% vs. 92.4% | Hurt |
| Use LayerNorm instead of BatchNorm (ablation) | Aligned ResNet20 interpolation · CIFAR-10, 8× midpoint | accuracy: 32.3% -> 90.3% | Helped |
| Increase ResNet width (ablation) | ResNet20 · CIFAR-10 | loss barrier: below 0.05 | Helped |
| Use the closed-form approximate variant instead of forward-pass exact REPAIR (ablation) | REPAIR · ImageNet, ResNet50 interpolation | loss barrier | Hurt |
| Apply approximate rescaling to hidden units (ablation) | Aligned MLP interpolation · MNIST | accuracy: up to 27 layers ... over 90%; ... limit after only 6 layers | Helped |
Closest papers by meaning
- Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.84)
- Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion (ICML 2025 · similarity 0.83)
- The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions (ICML 2025 · similarity 0.83)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.82)
- On the Emergence of Cross-Task Linearity in Pretraining-Finetuning Paradigm (ICML 2024 · similarity 0.82)
- Understanding Layer Patching in Model Size Interpolation (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.