Research map / Vision-language and multimodal / CLIP and zero-shot
Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery
Andy V Huynh, Lauren Gillespie, Jael Lopez-Saucedo, Claire Tang, Rohan Sikand, Moisés Expósito-Alonso
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train with CRISP ground-aerial contrastive objective instead of ImageNet pre-training | ResNet50 species recognition · NMV species recognition (0.25% labels) | Top-1 accuracy: 19.57 -> 36.05 | Improved |
| CRISP pre-training versus ImageNet at the largest label fraction | ResNet50 species recognition · NMV species recognition (20% labels) | Top-1 accuracy: 74.90 -> 70.96 | Hurt |
| use learned weighting of the two loss directions (CRISP-Par) instead of symmetric loss (ablation) | CRISP · NMV species recognition (1% labels) | Top-1 accuracy: 50.55 -> 51.04 | Improved |
| use many-to-one multi-positive loss instead of standard CRISP loss (ablation) | CRISP · NMV species recognition (0.25% labels) | Top-1 accuracy: 36.05 -> 35.06 | Hurt |
| add remote sensing crop/flip/rotation augmentation to aerial images (ablation) | CRISP · NMV species recognition (0.25% labels) | Top-1 accuracy: 36.05 -> 35.07 | No clear effect |
| pre-train aerial encoder with CRISP instead of SatMAE/SauMoCo/random init | ResNet50 species distribution mapping · NMV SDM (0.25% labels) | Top-30 macro accuracy: 15.51 vs 8.12 (SauMoCo) | Improved |
| CRISP pre-training versus self-supervised baselines on plain Top-5 accuracy with more labels | ResNet50 species distribution mapping · NMV SDM (20% labels) | Top-5 accuracy: 6.91 vs 14.76 (SatMAE-NMV) | Hurt |
| CRISP pre-training on rare species in aerial SDM | ResNet50 species distribution mapping · NMV SDM (0.25% labels) | Top-30-R accuracy: 6.8% vs 0.27% (average baseline) | Improved |
| add aerial encoder via mixture of experts to ground-level species classifier | CRISP ground encoder · NMV species recognition | Top-1 accuracy: 24.1% decrease | Hurt |
| CRISP pre-training versus Tile2Vec, SauMoCo, random features for crop mapping | frozen NAIP features with LR/MLP/RF · Crop type mapping (10,000 examples, logistic regression) | accuracy: 63.90 -> 67.75 (vs SauMoCo) | Improved |
| many-to-one objective with fewer training examples (ablation) | CRISP-M2o · Crop type mapping (1,000 examples, logistic regression) | accuracy: 66.20 (10,000) -> 57.54 (1,000) | Hurt |
| pre-train with CRISP instead of MoCo-v2/SauMoCo or ImageNet/random on Auto Arborist | Auto Arborist tree genus identification · Auto Arborist (3 held-out cities) | Top-5 macro accuracy: 16.33 -> 22.07 (Santa Monica, ground) | Improved |
Closest papers by meaning
- Benchmarking Representation Learning for Natural World Image Collections (CVPR 2021 · similarity 0.88)
- DUNIA: Pixel-Sized Embeddings via Cross-Modal Alignment for Earth Observation Applications (ICML 2025 · similarity 0.86)
- CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale (ICLR 2025 · similarity 0.85)
- BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning (NeurIPS 2025 · similarity 0.85)
- INQUIRE: A Natural World Text-to-Image Retrieval Benchmark (NeurIPS 2024 · similarity 0.85)
- DiffCLIP: Few-shot Language-driven Multimodal Classifier (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
