Research map / Learning theory and algorithms / Combinatorial and search optimization
DB-KSVD: Scalable Alternating Optimization for Disentangling High-Dimensional Embedding Spaces
Romeo Valentin, Sydney Michelle Katz, Vincent Vanhoucke, Mykel Kochenderfer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DB-KSVD instead of classic KSVD (scalable matching pursuit, batched updates, Lanczos) | KSVD · Gemma-2-2B embeddings, 2.6M samples, m=4096, k=20 | runtime to converge: over 30 days -> approximately 8 minutes | Improved |
| add Matryoshka structuring to DB-KSVD (ablation) | DB-KSVD · Gemma-2-2B SAEBench (m=4096) | autointerpretability and absorption | Improved |
| add Matryoshka structuring to DB-KSVD (ablation) | DB-KSVD · Gemma-2-2B SAEBench (m=4096) | loss recovered and SCR | Hurt |
| add Matryoshka structuring to DB-KSVD (ablation) | DB-KSVD · Gemma-2-2B dictionaries (m=4096) | dictionary coherence | Improved |
| use DB-KSVD instead of Standard ReLU SAE | Gemma-2-2B dictionary (m=4096) · SAEBench | SAEBench metrics | Mixed |
| use DB-KSVD instead of SAE on DINOv2-B | dictionary learning · DINOv2-B ImageNet-1k, k=32 | sparse probing accuracy: 0.932 -> 0.893 | Hurt |
| use DB-KSVD instead of SAE on DINOv2-B | dictionary learning · DINOv2-B ImageNet-1k, k=32 | variance explained: 0.794 -> 0.817 | Improved |
| offload matrix products to GPU (CPU+GPU vs CPU only) (ablation) | DB-KSVD · Gemma-2-2B, m=4096, k=20 | runtime (s): 5.47 -> 11.43 | Hurt |
| increase sparsity k from 40 to 80 at small dictionary size (ablation) | DB-KSVD (CPU) · Gemma-2-2B, m=4096 | runtime (s): 10.86 -> 337.63 | Hurt |
Closest papers by meaning
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.86)
- Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (ICML 2025 · similarity 0.86)
- Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation (ICML 2025 · similarity 0.85)
- Efficient Dictionary Learning with Switch Sparse Autoencoders (ICLR 2025 · similarity 0.85)
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.85)
- SASA: Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
