Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Function-Space Regularization in Neural Networks: A Probabilistic Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add empirical function-space regularization to parameter-space MAP training | ResNet-18 on FashionMNIST · FashionMNIST | NLL: 0.26 \pms{0.00} -> 0.19 \pms{0.00} | Helped |
| Add empirical function-space regularization to parameter-space MAP training | ResNet-18 on FashionMNIST · FashionMNIST | ECE: 3.6\% \pms{0.0} -> 1.8\% \pms{0.1} | Helped |
| Add empirical function-space regularization to parameter-space MAP training | ResNet-18 on CIFAR-10 · CIFAR-10 | accuracy: 94.9\% \pms{0.2} -> 95.1\% \pms{0.1} | Helped |
| Add empirical function-space regularization to parameter-space MAP training | ResNet-18 on CIFAR-10 · CIFAR-10 | ECE: 3.0\% \pms{0.1} -> 2.1\% \pms{0.1} | Helped |
| Use FS-EB instead of PS-MAP | ResNet-18 semantic shift detection · FashionMNIST vs MNIST | OOD AUROC: 94.9\% \pms{0.4} -> 99.9\% \pms{0.0} | Helped |
| Use FS-EB instead of PS-MAP | ResNet-18 semantic shift detection · CIFAR-10 vs SVHN | OOD AUROC: 93.0\% \pms{0.4} -> 99.4\% \pms{0.1} | Helped |
| Use FS-EB for ImageNet-pretrained transfer learning | ResNet-18 on CIFAR-10 · CIFAR-10 (ImageNet-pretrained) | accuracy: 96.2\% \pms{0.1} -> 96.2\% \pms{0.1} | No clear effect |
| Use FS-EB for ImageNet-pretrained transfer learning | ResNet-18 on CIFAR-10 · CIFAR-10 (ImageNet-pretrained) | ECE: 3.2\% \pms{0.2} -> 1.3\% \pms{0.1} | Helped |
| Use FS-EB instead of PS-MAP | ResNet-18 evaluated on CIFAR-10.1 · CIFAR-10.1 | accuracy: 88.0\% \pm 0.1 -> 86.8\% \pm 0.4 | Hurt |
| Use FS-EB instead of PS-MAP | ImageNet-pretrained ResNet-50 · APTOS | NLL: 0.83 -> 0.78 | Helped |
| Use a deep ensemble | FS-EB on FashionMNIST · FashionMNIST | OOD AUROC | No clear effect |
| Use training inputs instead of the benchmark context dataset (ablation) | FS-EB on CIFAR-10 · CIFAR-10 | OOD AUROC: 99.4\% \pms{0.1} -> 92.1\% \pms{0.6} | Hurt |
Closest papers by meaning
- Learning Neural Networks with Adaptive Regularization (NeurIPS 2019 · similarity 0.89)
- Function-space Parameterization of Neural Networks for Sequential Learning (ICLR 2024 · similarity 0.88)
- Network-to-Network Regularization: Enforcing Occam's Razor to Improve Generalization (NeurIPS 2021 · similarity 0.88)
- Continual Learning via Sequential Function-Space Variational Inference (ICML 2022 · similarity 0.87)
- Tractable Function-Space Variational Inference in Bayesian Neural Networks (NeurIPS 2022 · similarity 0.87)
- Improving Adaptivity via Over-Parameterization in Sequence Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.