Research map / Label-efficient and robust learning / Self-supervised and contrastive
Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CST instead of HSNet with image-level supervision | WFS-CS · WFS-CS, image-level supervision | segmentation mIoU (%): 20.1 -> 33.2 | Helped |
| Use CST instead of ASNet with image-level supervision | WFS-CS · WFS-CS, image-level supervision | segmentation mIoU (%): 15.6 -> 33.2 | Helped |
| Use CST instead of HSNet with pixel-level supervision | WFS-CS · WFS-CS, pixel-level supervision | segmentation mIoU (%): 49.5 -> 55.5 | Helped |
| Use CST instead of ASNet with pixel-level supervision | WFS-CS · WFS-CS, pixel-level supervision | segmentation mIoU (%): 53.7 -> 55.5 | Helped |
| Use CST with image-level labels instead of Tian et al.'s pixel-level method | GFS-Seg on Pascal-5i · 1-shot, novel classes | segmentation mIoU (%): 18.9 -> 28.7 | Helped |
| Use pixel-level rather than image-level supervision | CST on Pascal-5i GFS-Seg · 1-shot, novel classes | segmentation mIoU (%): 28.7 -> 50.3 | Helped |
| Use pixel-level rather than image-level supervision | CST on Pascal-5i GFS-Seg · 5-shot, novel classes | segmentation mIoU (%): 33.5 -> 58.9 | Helped |
| Train CST over query-support cross-correlated features to discover support-relevant semantics | CST · one-shot WFS-CS with zero GT mask labels | mIoU: 16.0 -> 33.2 | Helped |
Closest papers by meaning
- A Closer Look at Self-Supervised Lightweight Vision Transformers (ICML 2023 · similarity 0.90)
- Efficient Self-supervised Vision Transformers for Representation Learning (ICLR 2022 · similarity 0.89)
- Patch-Level Representation Learning for Self-Supervised Vision Transformers (CVPR 2022 · similarity 0.89)
- A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity (ICLR 2023 · similarity 0.89)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.88)
- Self-Supervised Learning of Intertwined Content and Positional Features for Object Detection (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.