Research map / Architectures and efficiency / RNNs and normalization
Hardware Resilience Properties of Text-Guided Image Classifiers
Syed Talal Wasim, Kabila Haile Soboka, Abdulrahman Mahmoud, Salman Khan, David Brooks, Gu-Yeon Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize projection layer with CLIP embeddings of GPT-3 descriptions (ablation) | ResNet-50 · ImageNet val | last-layer reliability improvement: 6.09x vs baseline (random init 1.74x) | Improved |
| Use hand-crafted CLIP prompt init instead of random init (ablation) | ResNet-50 · ImageNet val | last-layer reliability improvement: 1.74 -> 5.06 | Improved |
| Use GPT-3 descriptions instead of simple CLIP prompt for init (ablation) | ResNet-50 · ImageNet val | last-layer reliability improvement: 5.06 -> 6.09 | Improved |
| Use GPT-3 descriptions instead of simple CLIP prompt for init (ablation) | ResNet-50 · ImageNet val | reliability improvement across backbone: 3.28 -> 3.93 | Improved |
| Random init of projection layer still gives some gain (ablation) | ResNet-50 · ImageNet val | last-layer reliability improvement: 1.09 across backbone, 1.74 last layer | Mixed |
| Apply text-guided classifier across backbones | VGG family · ImageNet val | last-layer reliability improvement: 13.68x average | Improved |
| Apply text-guided classifier across backbones | ResNet family · ImageNet val | last-layer reliability improvement: 4.01x average | Improved |
| Apply text-guided classifier to Swin-V2-T | Swin-V2-T · ImageNet val | overall reliability improvement: 1.07x | No clear effect |
| Apply text-guided classifier | ResNet-101 · ImageNet val | top-1 accuracy: 77.25 -> 75.52 | Hurt |
| Apply text-guided classifier | MaxViT-T · ImageNet val | top-1 accuracy: 82.98 -> 83.08 | No clear effect |
| Apply text-guided classifier | ResNet-50 · ImageNet val | Top2Diff: 70.32 -> 73.67 | Improved |
| Apply text-guided classifier on CIFAR100 | FocalNet-T · CIFAR100 | top-1 accuracy: 77.06 -> 79.21 | Improved |
Closest papers by meaning
- ChatGPT-Powered Hierarchical Comparisons for Image Classification (NeurIPS 2023 · similarity 0.83)
- ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training (TPAMI 2022 · similarity 0.83)
- Multi-Scale Dense Networks for Resource Efficient Image Classification (ICLR 2018 · similarity 0.83)
- Compound Text-Guided Prompt Tuning via Image-Adaptive Cues (AAAI 2024 · similarity 0.82)
- Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels? (ICCV 2023 · similarity 0.82)
- Why are Visually-Grounded Language Models Bad at Image Classification? (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
