Research map / Continual learning / Online and replay-based
Visualizing the PHATE of Neural Networks
Scott Gigante, Adam S Charles, Smita Krishnaswamy, Gal Mishne
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the multislice kernel with PHATE instead of standard visualization methods | Hidden-unit training traces on MNIST · MNIST | intraslice neighborhood preservation: 0.05 -> 0.26 (k=10); 0.21 -> 0.45 (k=40) | Improved |
| Use the multislice kernel with PHATE instead of the strongest comparator | Hidden-unit training traces on MNIST · MNIST | interslice neighborhood preservation: 0.96 -> 0.95 (k=10); 0.94 -> 0.93 (k=40) | Hurt |
| Use multislice PHATE instead of the strongest comparator | Hidden-unit training traces on MNIST · MNIST | loss correlation: 0.47 -> 0.81 | Improved |
| Use Naive Rehearsal instead of Adam | 2-layer MLP continual learning · Split MNIST task learning | validation loss: 0.042 -> 0.047 | Hurt |
| Use Adagrad instead of Adam | 2-layer MLP continual learning · Split MNIST domain learning | validation loss: 1.062 -> 0.462 | Improved |
| Use Adagrad instead of Adam | 2-layer MLP continual learning · Split MNIST class learning | validation loss: 4.156 -> 1.884 | Improved |
| Preserve hidden-unit neighborhood structure across task switches | Continual-learning networks · Split MNIST continual learning | validation loss: ARI and final validation loss have correlation rho = 0.94 | Improved |
| Apply dropout rather than vanilla training | 3-layer MLP · MNIST generalization | memorization error: 0.04 -> -0.09 | Improved |
| Apply activity L1 regularization rather than vanilla training | 3-layer MLP · MNIST generalization | memorization error: 0.04 -> 0.11 | Hurt |
| Apply activity L2 regularization rather than vanilla training | 3-layer MLP · MNIST generalization | memorization error: 0.04 -> 0.12 | Hurt |
| Use training data alone to construct the visualization | M-PHATE · MNIST generalization | Spearman correlation between embedding variance and memorization: rho=-0.98 -> rho = -0.95 | No clear effect |
Closest papers by meaning
- Don't trust your eyes: on the (un)reliability of feature visualizations (ICML 2024 · similarity 0.86)
- Neuro-Visualizer: A Novel Auto-Encoder-Based Loss Landscape Visualization Method With an Application in Knowledge-Guided Machine Learning (ICML 2024 · similarity 0.85)
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.85)
- Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking (NeurIPS 2025 · similarity 0.84)
- Learning Reveals Invisible Structure in Low-Rank RNNs (NeurIPS 2026 · similarity 0.84)
- Three Mechanisms of Feature Learning in a Linear Network (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
