Research map / Learning theory and algorithms / Matrix and tensor methods
Learning Representation from Neural Fisher Kernel with Low-rank Approximation
Ruixiang ZHANG, Shuangfei Zhai, Etai Littwin, Joshua M. Susskind
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a 128-dimensional NFK embedding instead of GAN activations | GAN representation for linear evaluation · CIFAR-10 | accuracy: 65.3 -> 89.8 | Improved |
| Compress GAN Fisher vectors into a 128-dimensional NFK embedding | GAN representation for linear evaluation · CIFAR-10 | accuracy: 89.8 -> 89.8 | No clear effect |
| Increase VAE NFK embedding from 128 to 256 dimensions (ablation) | VAE-NFK · CIFAR-10 | accuracy: 63.2 -> 68.7 | Improved |
| Increase GAN NFK embedding from 128 to 256 dimensions (ablation) | GAN-NFK · CIFAR-10 | accuracy: 89.8 -> 89.8 | No clear effect |
| Use a pretrained GAN's NFK-128d representation with labeled-only linear probing | NFK-128d · CIFAR-10 semi-supervised classification | error rate: 20.68 vs 9.65 (500 labels); 14.77 vs 7.75 (1000 labels); 13.82 vs 7.03 (2000 labels); 12.95 vs 6.24 (4000 labels) | Mixed |
| Use a pretrained GAN's NFK-128d representation with labeled-only linear probing | NFK-128d · SVHN semi-supervised classification | error rate: 8.74 vs 3.64 (500 labels); 4.47 vs 3.27 (1000 labels); 3.82 vs 3.04 (2000 labels); 3.19 vs 2.89 (4000 labels) | Mixed |
| Distill with the teacher's NFK representation rather than layerwise targets | WRN-40-2 teacher → WRN-16-1 student · CIFAR-10 knowledge distillation | accuracy: 91.85 -> 92.42 | Improved |
Closest papers by meaning
- Non-parametric Representation Learning with Kernels (AAAI 2024 · similarity 0.87)
- Adversarial Fisher Vectors for Unsupervised Representation Learning (NeurIPS 2019 · similarity 0.86)
- A Representer Theorem for Deep Kernel Learning (JMLR 2019 · similarity 0.85)
- Convolutional Deep Kernel Machines (ICLR 2024 · similarity 0.85)
- Hierarchically Compositional Kernels for Scalable Nonparametric Learning (JMLR 2017 · similarity 0.85)
- Scalable Neural Network Kernels (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
