AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery

Andy V Huynh, Lauren Gillespie, Jael Lopez-Saucedo, Claire Tang, Rohan Sikand, Moisés Expósito-Alonso

ECCV 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: Recaptioning and text-supervised pretraining · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
pre-train with CRISP ground-aerial contrastive objective instead of ImageNet pre-trainingResNet50 species recognition · NMV species recognition (0.25% labels)Top-1 accuracy: 19.57 -> 36.05Improved
CRISP pre-training versus ImageNet at the largest label fractionResNet50 species recognition · NMV species recognition (20% labels)Top-1 accuracy: 74.90 -> 70.96Hurt
use learned weighting of the two loss directions (CRISP-Par) instead of symmetric loss (ablation)CRISP · NMV species recognition (1% labels)Top-1 accuracy: 50.55 -> 51.04Improved
use many-to-one multi-positive loss instead of standard CRISP loss (ablation)CRISP · NMV species recognition (0.25% labels)Top-1 accuracy: 36.05 -> 35.06Hurt
add remote sensing crop/flip/rotation augmentation to aerial images (ablation)CRISP · NMV species recognition (0.25% labels)Top-1 accuracy: 36.05 -> 35.07No clear effect
pre-train aerial encoder with CRISP instead of SatMAE/SauMoCo/random initResNet50 species distribution mapping · NMV SDM (0.25% labels)Top-30 macro accuracy: 15.51 vs 8.12 (SauMoCo)Improved
CRISP pre-training versus self-supervised baselines on plain Top-5 accuracy with more labelsResNet50 species distribution mapping · NMV SDM (20% labels)Top-5 accuracy: 6.91 vs 14.76 (SatMAE-NMV)Hurt
CRISP pre-training on rare species in aerial SDMResNet50 species distribution mapping · NMV SDM (0.25% labels)Top-30-R accuracy: 6.8% vs 0.27% (average baseline)Improved
add aerial encoder via mixture of experts to ground-level species classifierCRISP ground encoder · NMV species recognitionTop-1 accuracy: 24.1% decreaseHurt
CRISP pre-training versus Tile2Vec, SauMoCo, random features for crop mappingfrozen NAIP features with LR/MLP/RF · Crop type mapping (10,000 examples, logistic regression)accuracy: 63.90 -> 67.75 (vs SauMoCo)Improved
many-to-one objective with fewer training examples (ablation)CRISP-M2o · Crop type mapping (1,000 examples, logistic regression)accuracy: 66.20 (10,000) -> 57.54 (1,000)Hurt
pre-train with CRISP instead of MoCo-v2/SauMoCo or ImageNet/random on Auto ArboristAuto Arborist tree genus identification · Auto Arborist (3 held-out cities)Top-5 macro accuracy: 16.33 -> 22.07 (Santa Monica, ground)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy