Research map / Image generation / Fast diffusion sampling
DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights
Saumya Gupta, Scott Biggs, Moritz Laber, Zohair Shafi, Robin Walters, Ayan Paul
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Git Re-Basin canonicalization of training weights when flow model capacity is low (ablation) | DeepWeightFlow · MNIST MLP (flow hidden dim 64) | accuracy (%): 25.54 -> 57.80 | Improved |
| add TransFusion canonicalization when flow model capacity is low (ablation) | DeepWeightFlow · CIFAR-10 ViT-Small-192 (flow hidden dim 64) | accuracy (%): 12.67 -> 43.13 | Improved |
| add Git Re-Basin canonicalization at sufficient flow capacity (ablation) | DeepWeightFlow · MNIST MLP (flow hidden dim 512) | accuracy (%): 96.19 -> 96.17 | No clear effect |
| add Git Re-Basin canonicalization at sufficient capacity (ablation) | DeepWeightFlow · CIFAR-10 ResNet-18 (complete weights) | accuracy (%): 93.47 -> 93.55 | No clear effect |
| add Re-Basin canonicalization (ablation) | DeepWeightFlow · STL-10 ResNet-18 | accuracy (%): 62.50 -> 62.46 | No clear effect |
| add TransFusion canonicalization at larger flow capacity (ablation) | DeepWeightFlow · CIFAR-10 ViT-Small-192 | accuracy (%): 82.58 -> 83.07 | Improved |
| recalibrate BN running statistics on generated weights instead of copying reference statistics (ablation) | DeepWeightFlow · CIFAR-10 ResNet-18 (with Re-Basin) | accuracy (%): 19.06 -> 93.05 | Improved |
| recalibrate BN running statistics (ablation) | DeepWeightFlow · CIFAR-10 ResNet-20 (with Re-Basin) | accuracy (%): 17.88 -> 74.57 | Improved |
| add incremental PCA compression of training weights (ablation) | DeepWeightFlow · CIFAR-10 ResNet-20 | training time (min): 11.25 -> 1.23 | Improved |
| add incremental PCA compression of training weights (ablation) | DeepWeightFlow · CIFAR-10 ResNet-20 | accuracy (%): 75.07 -> 75.96 | No clear effect |
| use Gaussian source distribution instead of Kaiming initialization (ablation) | DeepWeightFlow · MNIST MLP | accuracy / diversity | Improved |
| use DeepWeightFlow generated weights as transfer-learning initialization instead of FLoWN | ResNet-18 CIFAR-10 -> STL-10 · STL-10 (zero-shot) | accuracy (%): 35.16 -> 48.32 | Improved |
Closest papers by meaning
- Diffusion-based Neural Network Weights Generation (ICLR 2025 · similarity 0.88)
- HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories (CVPR 2025 · similarity 0.86)
- Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks (ICLR 2025 · similarity 0.86)
- On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity (NeurIPS 2025 · similarity 0.86)
- The Effect of Weight Precision on the Neuron Count in Deep ReLU Networks (ICML 2024 · similarity 0.85)
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
