Research map / Vision-language and multimodal / CLIP and zero-shot
FILIP: Fine-grained Interactive Language-Image Pre-Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace global image-text similarity with cross-modal late interaction (ablation) | ViT-B/32 · MSCOCO zero-shot image-to-text retrieval (YFCC100M subset) | R@1: 25.0 -> 30.5 | Helped |
| Add image augmentation (ablation) | ViT-B/32 · MSCOCO zero-shot image-to-text retrieval (YFCC100M subset) | R@1: 25.0 -> 26.1 | Helped |
| Add back translation (ablation) | ViT-B/32 with image augmentation · MSCOCO zero-shot image-to-text retrieval (YFCC100M subset) | R@1: 26.1 -> 29.2 | Helped |
| Combine image augmentation, back translation, and late interaction (ablation) | FILIP base · ImageNet zero-shot image classification (YFCC100M subset) | top-1 accuracy: 30.4 -> 37.8 | Helped |
| Reduce last-layer feature precision from fp32 to fp16 (ablation) | Late interaction with 512-dimensional embeddings and all tokens · ImageNet zero-shot image classification | top-1 accuracy: 34.6 -> 34.5 | Hurt |
| Reduce embedding dimension from 512 to 256 (ablation) | Late interaction with fp16 features and all tokens · ImageNet zero-shot image classification | top-1 accuracy: 34.5 -> 35.2 | Helped |
| Keep 25% of tokens instead of all tokens for late interaction (ablation) | Late interaction with 256-dimensional fp16 features · ImageNet zero-shot image classification | top-1 accuracy: 35.2 -> 34.3 | Hurt |
| Use FILIP late interaction rather than global cosine similarity | Zero-shot image-text retrieval and classification · Flickr30K zero-shot text-to-image retrieval | R@1: 68.7 -> 75.0; ALIGN: 75.7 | Mixed |
| Use FILIP rather than CLIP with the ViT-L/14 architecture | ViT-L/14 · 12-dataset zero-shot image classification average | top-1 accuracy: 75.3 -> 78.3 | Helped |
Closest papers by meaning
- FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputs (CVPR 2025 · similarity 0.91)
- MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization (ICML 2024 · similarity 0.89)
- Improving fine-grained understanding in image-text pre-training (ICML 2024 · similarity 0.89)
- FineCLIP: Self-distilled Region-based CLIP for Better Fine-grained Understanding (NeurIPS 2024 · similarity 0.89)
- FG-CLIP: Fine-Grained Visual and Textual Alignment (ICML 2025 · similarity 0.89)
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.