Research map / Vision-language and multimodal / Cross-modal learning
Learning multimodal dictionary decompositions with group-sparse autoencoders
Chiraag Kaushik, Davis Barch, Andrea Fanelli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add group-sparsity loss (ablation) | TopK SAE (CLIP) · CIFAR-10 zero-shot | accuracy: 0.657 -> 0.808 | Improved |
| add group-sparsity loss (ablation) | TopK SAE (CLIP) · CIFAR-100 zero-shot | accuracy: 0.418 -> 0.526 | Improved |
| add group-sparsity loss (ablation) | TopK SAE (CLIP) · ImageNet zero-shot | accuracy: 0.303 -> 0.354 | Improved |
| add cross-modal random masking to GSAE (ablation) | GSAE (CLIP) · CIFAR-10 zero-shot | accuracy: 0.808 -> 0.842 | Improved |
| add cross-modal random masking to GSAE (ablation) | GSAE (CLIP) · ImageNet zero-shot | accuracy: 0.354 -> 0.373 | Improved |
| add group-sparsity loss (ablation) | TopK SAE (CLAP) · GTZAN Genres zero-shot | accuracy: 0.376 -> 0.705 | Improved |
| add cross-modal random masking to GSAE (ablation) | GSAE (CLAP) · GTZAN Genres zero-shot | accuracy: 0.705 -> 0.672 | Hurt |
| add cross-modal random masking to GSAE (ablation) | GSAE (CLAP) · FMACaps retrieval | MRR: 0.050 -> 0.061 | Improved |
| add group-sparsity loss and masking (ablation) | TopK SAE (SigLIP2) · ImageNet zero-shot | accuracy: 0.451 -> 0.481 | Improved |
| add masking to GSAE (ablation) | GSAE (SigLIP2) · CIFAR-10 zero-shot | accuracy: 0.922 -> 0.906 | No clear effect |
| add group-sparsity loss and masking (ablation) | TopK SAE (AIMv2) · ImageNet zero-shot | accuracy: 0.443 -> 0.522 | Improved |
| add group-sparsity loss and masking (ablation) | TopK SAE (MS COCO) · CIFAR-10 zero-shot | accuracy: 0.547 -> 0.820 | Improved |
Closest papers by meaning
- Interpreting CLIP with Hierarchical Sparse Autoencoders (ICML 2025 · similarity 0.89)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.89)
- Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE) (NeurIPS 2024 · similarity 0.88)
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.88)
- A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders (NeurIPS 2025 · similarity 0.88)
- Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
