Research map / Vision-language and multimodal / Cross-modal learning
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a mixture-of-experts posterior instead of the product-of-experts posterior | Multimodal VAE · MNIST-SVHN cross-generation | Digit matching (%): 9.5 -> 86.4 (MNIST to SVHN); 9.3 -> 69.1 (SVHN to MNIST) | Helped |
| Use a mixture-of-experts posterior instead of the product-of-experts posterior | Multimodal VAE · MNIST-SVHN joint generation | Digit matching (%): 12.7 -> 42.1 | Helped |
| Use a mixture-of-experts posterior instead of the product-of-experts posterior | Multimodal VAE · CUB Image-Captions | Joint image-caption correlation: -0.095 -> 0.263 | Helped |
| Use a mixture-of-experts posterior instead of the product-of-experts posterior | Multimodal VAE · MNIST-SVHN joint log likelihood | log likelihood: 2961.80 -> 6261.40 | Helped |
| Provide both modalities to the MVAE rather than MNIST alone | MVAE · MNIST-SVHN | MNIST digit classification accuracy (%): 95.7 -> 94.9 | Hurt |
| Provide both modalities to the MVAE rather than SVHN alone | MVAE · MNIST-SVHN | SVHN digit classification accuracy (%): 9.1 -> 90.1 | Helped |
| Use the tighter alternative MoE IWAE bound (ablation) | MMVAE · MNIST-SVHN joint generation | Digit matching (%): 42.1 -> 24.2 | Hurt |
| Use the tighter alternative MoE IWAE bound (ablation) | MMVAE · MNIST to SVHN | Digit matching (%): 86.4 -> 74.6 | Hurt |
| Use the tighter alternative MoE IWAE bound (ablation) | MMVAE · SVHN to MNIST | Digit matching (%): 69.1 -> 15.6 | Hurt |
| Add multimodal observations to single-modality learning | MMVAE · MNIST-SVHN, MNIST reconstruction | log likelihood: 868.37 -> 868.76 | No clear effect |
| Add multimodal observations to single-modality learning | MVAE · MNIST-SVHN, MNIST reconstruction | log likelihood: -107.46 -> -176.68 | Hurt |
Closest papers by meaning
- MMVAE+: Enhancing the Generative Quality of Multimodal VAEs without Compromises (ICLR 2023 · similarity 0.91)
- Multimodal Generative Models for Scalable Weakly-Supervised Learning (NeurIPS 2018 · similarity 0.91)
- Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoders (ICML 2025 · similarity 0.90)
- On the Limitations of Multimodal VAEs (ICLR 2022 · similarity 0.89)
- InfoVAE: Balancing Learning and Inference in Variational Autoencoders (AAAI 2019 · similarity 0.89)
- Unity by Diversity: Improved Representation Learning for Multimodal VAEs (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.