Research map / Label-efficient and robust learning / Recognition and long-tail
Recovery and Generalization in Over-Realized Dictionary Learning
Jeremias Sulam, Chong You, Zhihui Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase dictionary size from p=70 to p'>70 (up to 500) during ODL+OMP learning | Online Dictionary Learning (ODL) with OMP · Synthetic 50x70 dictionary, k=3, 300 training samples | train/test risk | Improved |
| increase dictionary size from p=70 to p'>70 (up to 500) during ODL+OMP learning | Online Dictionary Learning (ODL) with OMP · Synthetic 50x70 dictionary, k=3, 300 training samples | recovery distance to ground truth | Improved |
| keep increasing dictionary size well beyond the ground-truth size with fixed 300 samples | ODL over-realized dictionary · Synthetic 50x70 dictionary, k=3, 300 training samples | generalization error | Hurt |
| increase number of training samples for a given over-realized dictionary size | ODL over-realized dictionary · Synthetic 50x70 dictionary, k=3 | risk and recovery error | Improved |
| distill over-realized dictionary to p atoms by keeping the p most frequently used atoms | ODL+OMP over-realized dictionary (100 atoms, 500 samples) · Synthetic 50x70 dictionary, k=3 | risk and dictionary error | Improved |
| over-realization plus distillation across sparsity, dimension and redundancy settings | ODL+OMP · Synthetic dictionaries with varied parameters | risk and log10 dictionary error | Improved |
| use ODL with l1 regularizer (Lasso coding) instead of OMP | over-realization plus distillation · Synthetic dictionaries | dictionary recovery error | Improved |
| use batch K-SVD instead of ODL | over-realization plus distillation · Synthetic dictionaries | dictionary recovery error | Improved |
| add Gaussian noise (variance sigma^2) to training samples | over-realization and distillation · Synthetic dictionaries with noise | relative improvement in recovery over traditional learning | Mixed |
| retrain from initialization using only the atoms found by distillation (ablation) | over-realized dictionary learning · Synthetic dictionaries | recovery performance | Hurt |
Closest papers by meaning
- On Learning Sparsely Used Dictionaries from Incomplete Samples (ICML 2018 · similarity 0.83)
- Fast greedy algorithms for dictionary selection with generalized sparsity constraints (NeurIPS 2018 · similarity 0.83)
- On the Sparsity-Storage-Accuracy Tradeoff in Parsimoniously Activated Dictionary Learning (NeurIPS 2026 · similarity 0.81)
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.80)
- Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures (NeurIPS 2026 · similarity 0.80)
- Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (ICML 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
