Research map / Vision-language and multimodal / Cross-modal learning
ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology
Srikumar Sastry, Subash Khanal, Aayush Dhakal, Jiayu Lin, Dan Cher, Phoenix Jarosz, Nathan Jacobs
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train probabilistic masked embedding model on top of TaxaBind encoders | TaxaBind · TaxaBench-8k zero-shot classification | accuracy: 34.45 -> 39.45 | Improved |
| same | TaxaBind · iNat-2021 zero-shot classification | accuracy: 70.09 -> 75.83 | Improved |
| same | BioTroveCLIP · CUB-200-2011 | accuracy: 82.41 -> 82.85 | Improved |
| same | TaxaBind · BirdCLEF'23 audio linear probe | top-1 accuracy: 42.19 -> 52.30 | Improved |
| same with multimodal input | TaxaBind · BirdCLEF'23 audio linear probe (multimodal) | top-1 accuracy: 46.97 -> 59.06 | Improved |
| replace contrastive reconstruction loss with MSE (ablation) | ProM3E · TaxaBench-8k classification | accuracy: 39.45 -> 33.92 | Hurt |
| use contrastive instead of MSE loss (ablation) | ProM3E · EcoRegions linear probing | accuracy: 80.27 -> 81.35 | Improved |
| use 1 encoder layer instead of 4 (ablation) | ProM3E · TaxaBench-8k classification | accuracy: 38.36 -> 39.45 | Improved |
| raise encoder dim from 256 to 1024 (ablation) | ProM3E · EcoRegions linear probing | accuracy: 74.24 -> 81.35 | Improved |
| raise encoder dim from 1024 to 2048 (ablation) | ProM3E · TaxaBench-8k classification | accuracy: 39.45 -> 39.23 cls; 81.35 -> 82.52 lin | Mixed |
| use 4 register tokens instead of 0 (ablation) | ProM3E · EcoRegions linear probing | accuracy: 78.51 -> 81.35 | Improved |
| use 4 register tokens instead of 0 (ablation) | ProM3E · TaxaBench-8k classification | accuracy: 39.21 -> 39.45 | No clear effect |
Closest papers by meaning
- Unity by Diversity: Improved Representation Learning for Multimodal VAEs (NeurIPS 2024 · similarity 0.86)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.86)
- Aligning Multimodal Representations through an Information Bottleneck (ICML 2025 · similarity 0.85)
- INQUIRE: A Natural World Text-to-Image Retrieval Benchmark (NeurIPS 2024 · similarity 0.85)
- An Information Criterion for Controlled Disentanglement of Multimodal Data (ICLR 2025 · similarity 0.85)
- Scaling Language-centric Omnimodal Representation Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
