Research map / Probabilistic and scientific modeling / Variational and generative sampling
Relative gradient optimization of the Jacobian term in unsupervised deep learning
Luigi Gresele, Giancarlo Fissore, Adrián Javaloy, Bernhard Schölkopf, Aapo Hyvarinen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use relative-gradient updates instead of ordinary gradients to optimize the Jacobian term | Fully connected invertible neural networks · Gradient computation experiment; data dimensions from 10 to beyond 20,000 | gradient computation time: of the order of 100 | Improved |
| Use relative-gradient updates instead of ordinary gradients | Two-layer neural network trained on 1,000 datapoints of dimensionality 2 · Full-batch optimization | loss | No clear effect |
| Use the proposed relative-gradient density estimator instead of MADE MoG | Unconditional density estimation · POWER test | log-likelihood: 0.375\pm 0.013 -> 0.065\pm0.013 | Hurt |
| Use the proposed relative-gradient density estimator instead of Real NVP (10) | Unconditional density estimation · GAS test | log-likelihood: 8.357\pm 0.019 -> 6.978\pm0.020 | Hurt |
| Use the proposed relative-gradient density estimator instead of MADE MoG | Unconditional density estimation · HEPMASS test | log-likelihood: -18.368\pm 0.019 -> -21.958\pm0.019 | Hurt |
| Use the proposed relative-gradient density estimator instead of Real NVP (10) | Unconditional density estimation · MINIBOONE test | log-likelihood: -11.795\pm 0.453 -> -13.372\pm 0.450 | Hurt |
| Use the proposed relative-gradient density estimator instead of Real NVP (10) | Unconditional density estimation · BSDS300 test | log-likelihood: 153.28\pm 1.78 -> 151.12\pm0.28 | Hurt |
| Use the proposed relative-gradient density estimator instead of MADE MoG | Unconditional density estimation · MNIST test | log-likelihood: -1038.5 \pm 1.8 -> -1375.2 \pm 1.4 | Hurt |
Closest papers by meaning
- Self Normalizing Flows (ICML 2021 · similarity 0.85)
- Free Probability for predicting the performance of feed-forward fully connected neural networks (NeurIPS 2022 · similarity 0.85)
- Adaptive Proximal Gradient Methods for Structured Neural Networks (NeurIPS 2021 · similarity 0.85)
- Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (ICLR 2020 · similarity 0.85)
- Neglected Hessian component explains mysteries in sharpness regularization (NeurIPS 2024 · similarity 0.84)
- Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
