Research map / Optimization / Implicit bias and generalization
Improving Predictive State Representations via Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize gradient descent with spectral learning instead of random HMM-derived PSR (ablation) | PSR with contrastive-divergence SGD · Ring-topology HMM, 10,000 trajectories | test log loss | Helped |
| initialize with spectral learning instead of random (ablation) | PSR rank 20 · Wikipedia char-level | updates needed: 2 × 10^7 -> 5×10^5 | Helped |
| initialize with spectral learning instead of random (ablation) | PSR rank 20 · Wikipedia char-level | asymptotic test bpc | No clear effect |
| initialize with spectral learning instead of random (ablation) | PSR rank 1000 · Wikipedia char-level | test bpc: 3.5 -> ∼2.5 | Helped |
| add gradient descent refinement to spectrally learned PSR | spectral PSR rank 1000 · Wikipedia char-level | test bpc | Helped |
| add gradient refinement (CD objective) vs Shaban et al. exterior point refinement | spectral HMM-to-PSR · Unstructured HMM, 10 states | log loss | No clear effect |
| use our gradient refinement instead of Shaban et al. | spectral model · Ring-topology HMM, low rank | log loss | Helped |
| optimize conditional log loss instead of joint log loss (ablation) | PSR rank 1000 · Wikipedia char-level | descent speed | Helped |
Closest papers by meaning
- Completing State Representations using Spectral Learning (NeurIPS 2018 · similarity 0.92)
- Learning Predictive State Representations From Non-Uniform Sampling (AAAI 2018 · similarity 0.91)
- Learning invariant representations of time-homogeneous stochastic dynamical systems (ICLR 2024 · similarity 0.87)
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.87)
- SpectraLDS: Provable Distillation for Linear Dynamical Systems (NeurIPS 2025 · similarity 0.86)
- Spectral Alignment in Forward–Backward Representations via Temporal Abstraction (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.