Research map / Label-efficient and robust learning / Self-supervised and contrastive
SEPT: Towards Scalable and Efficient Visual Pre-training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select task-relevant unlabeled images by feature similarity instead of randomly sampling the same-sized pool | SEPT pre-training · classification tasks, 100k pre-training images | top-1 accuracy: 73.77 vs 98.73 (Flowers 5-shot); 23.91 vs 55.59 (Food101 5-shot) | Helped |
| increase selected pre-training data from 100k to 500k (ablation) | SEPT · Food101 5-shot | top-1 accuracy: 55.59 -> 69.72 | Helped |
| increase selected pre-training data from 500k to 1000k (ablation) | SEPT · Food101 5-shot | top-1 accuracy: 69.72 -> 72.17 | Helped |
| increase the unlabeled data pool from 55 million to 155 million images (ablation) | SEPT · Food101 and iNaturalist classification | top-1 accuracy: 54.82 -> 55.59 (Food101 5-shot); 27.46 -> 28.58 (iNat-I1k 5-shot) | No clear effect |
| use the 155m pool rather than YFCC-100m alone (ablation) | SEPT · Food101 5-shot | top-1 accuracy: 51.49 -> 55.59 | Helped |
| use ViT-L retrieval trained on IN22k rather than ViT-S retrieval trained on IN1k (ablation) | SEPT · Food101 5-shot | top-1 accuracy: 49.68 -> 53.64 | Helped |
| use instance crops instead of whole images as retrieval queries (ablation) | SEPT detection · CityScapes and VOC | detection performance: 29.9 -> 31.4 (CityScapes, 100k); 55.6 -> 57.4 (VOC, 100k) | Helped |
| use task-specific retrieval instead of random sampling | SEPT detection · CityScapes, VOC, and Logo-100 | detection performance: 26.3 -> 29.9; 47.0 -> 55.6; 55.3 -> 64.3 (100k, SEPT-Image) | Helped |
| use SEPT with 100k selected images instead of IN1k pre-training | SEPT · iNat-M1k 5-shot | top-1 accuracy: 36.40 -> 24.93 | Hurt |
| use SEPT with 100k selected images instead of IN1k pre-training | SEPT · Stanford Cars 10-shot | top-1 accuracy: 61.45 -> 59.52 | Hurt |
Closest papers by meaning
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.88)
- Rethinking Supervised Pre-Training for Better Downstream Transferring (ICLR 2022 · similarity 0.87)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.86)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.86)
- Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining (NeurIPS 2026 · similarity 0.85)
- Filter Images First, Generate Instructions Later: Pre-Instruction Data Selection for Visual Instruction Tuning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.