Research map / Label-efficient and robust learning / Test-time adaptation
Processing and acquisition traces in visual encoders: What does CLIP know about your camera?
Ryan Ramos, Vladan Stojnić, Giorgos Kordopatis-Zilos, Yuta Nakashima, Giorgos Tolias, Noa Garcia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use contrastive vision-language training instead of supervised or self-supervised | visual encoder embeddings · FlickrExif model (smart vs non-smart) | top-k neighbor same-camera rate: 59.0% to 70.0% vs at most 58.3% | Improved |
| use contrastive vision-language training | visual encoder embeddings · FlickrExif make prediction (95% masking) | linear probe accuracy: exceed 20% vs baseline of 11.11% | Improved |
| train CLIP with DINOv2-style color augmentations (ablation) | OpenCLIP ViT-B/32 · processing-label prediction on ImageNet/iNaturalist | linear probe accuracy | Improved |
| train CLIP with DINOv2-style color augmentations (ablation) | OpenCLIP ViT-B/32 · acquisition-label prediction on FlickrExif | linear probe accuracy | Mixed |
| train CLIP with color augmentations (ablation) | OpenCLIP ViT-B/32 · semantic kNN under coupled metadata | semantic accuracy | Improved |
| make metadata labels coincide with semantic labels (pos-same, neg-same) | CVL and ConvNeXt encoders · ImageNet/iNaturalist kNN | semantic accuracy | Hurt |
| apply different processing between train and test (all-diff, uniform) | visual encoders · ImageNet/iNaturalist kNN | semantic accuracy | No clear effect |
| make retrieval negatives share query camera type | CVL encoders · PairCams | recall@1: near-perfect recall@1 to below 0.85 | Hurt |
| increase center masking to 95% | visual encoders · ImageNet validation | semantic accuracy | Hurt |
| mask image content (95% vs none) when predicting acquisition labels | visual encoders · FlickrExif | acquisition accuracy | Hurt |
Closest papers by meaning
- Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions (NeurIPS 2025 · similarity 0.86)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.85)
- What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.85)
- Understanding Model Reprogramming for CLIP via Decoupling Visual Prompts (ICML 2025 · similarity 0.84)
- Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge (NeurIPS 2024 · similarity 0.84)
- UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
