Research map / Optimization / Implicit bias and generalization
Predictive Differential Training Guided by Training Dynamics
Fanqi Wang, Weisheng Tang, Landon Harris, Hairong Qi, Dan Wilson, Igor Mezic
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PDT (predictive differential training) | SGD on FCN · CIFAR-10 FCN | Time to baseline best train loss (s): 2174.32 -> 1313.52 | Improved |
| add PDT | AdamW on ViT-Huge · ImageNet-1K ViT-Huge | Time to baseline best train loss (s): 725564.86 -> 653854.05 | Improved |
| add PDT | SGD-M on ResNet-50 · ImageNet-1K ResNet-50 | Time to baseline best train loss (s): 110063.72 -> 88752.33 | Improved |
| add PDT | LAMB on AlexNet · CIFAR-10 AlexNet | Final accuracy: 0.8034 -> 0.8140 | Improved |
| add PDT | SGD with Momentum on AlexNet · CIFAR-10 AlexNet | Final accuracy: 0.6672 -> 0.7298 | Improved |
| add PDT | SGD on AlexNet · CIFAR-10 AlexNet | Final accuracy: 0.7930 -> 0.7978 | No clear effect |
| add PDT | SimSiam SGD Momentum · CIFAR-10 SimSiam | Final accuracy: 0.7285 -> 0.7685 | Improved |
| add PDT | SimSiam SGD Momentum · CIFAR-10 SimSiam | TTB-Loss (s): 9611.35 -> 4922.92 | Improved |
| replace dynamic masking with random subset of Koopman predictions (ablation) | PDT · CIFAR-10 AlexNet | training stability | Hurt |
| add PDT masking vs randomly raising learning rate of weight subset (ablation) | AlexNet SGD · CIFAR-10 AlexNet | training | Improved |
| switch prediction vs SGD by validation loss trend instead of masking (ablation) | PDT · CIFAR-10 AlexNet | validation loss | Hurt |
| increase prediction steps beyond nine (ablation) | PDT · CIFAR-10 AlexNet | training stability | Hurt |
Closest papers by meaning
- Identifying Equivalent Training Dynamics (NeurIPS 2024 · similarity 0.86)
- Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems (NeurIPS 2025 · similarity 0.85)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.85)
- KoNODE: Koopman-Driven Neural Ordinary Differential Equations with Evolving Parameters for Time Series Analysis (ICML 2025 · similarity 0.84)
- Conformal Online Learning of Deep Koopman Linear Embeddings (NeurIPS 2025 · similarity 0.84)
- Deep-Koopman-KANDy: Dictionary Discovery for Deep-Koopman Operators with Kolmogorov-Arnold Networks for Dynamics (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
