Research map / Architectures and efficiency / Attention and transformers
Discrete Representations Strengthen Vision Transformer Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add discrete and pixel embeddings to ViT-B | ViT-B · ImageNet-Rendition, Stylized-ImageNet, ImageNet-Sketch, ImageNet-C | accuracy / mCE: 38.15 -> 44.77; 10.39 -> 19.38; 28.60 -> 34.59; 53.51 -> 46.22 | Helped |
| Use discrete tokens only, omitting pixel embeddings (ablation) | ViT-B + Ours · ImageNet-Rendition, Stylized-ImageNet, ImageNet-Sketch | accuracy: 38.15 -> 48.82; 10.39 -> 22.19; 28.60 -> 39.10 | Helped |
| Use discrete tokens only, omitting pixel embeddings (ablation) | ViT-B + Ours · ImageNet-C | mCE: 46.22 -> 55.21 | Hurt |
| Remove positional embeddings (ablation) | Ours ViT-B · ImageNet, Stylized-ImageNet, ImageNet-C | accuracy / mCE: 79.48 -> 56.27; 19.38 -> 4.14; 46.22 -> 89.76 | Hurt |
| Increase codebook size from 1024 to 4096 (ablation) | Ours with quantized encoder · ImageNet-Rendition, ImageNet-C | accuracy / mCE: 45.92 -> 47.04; 58.64 -> 57.26 | Mixed |
| Increase codebook size from 4096 to 8192 (ablation) | Ours with quantized encoder · Stylized-ImageNet, ImageNet-C | accuracy / mCE: 21.33 -> 20.94; 57.26 -> 57.89 | Hurt |
| Replace concatenation with addition (ablation) | Ours ViT-B · ImageNet-Rendition, Stylized-ImageNet, ImageNet-C | accuracy / mCE: 44.77 -> 40.68; 19.38 -> 13.59; 46.22 -> 50.85 | Hurt |
| Replace concatenation with residual gating (ablation) | Ours ViT-B · ImageNet-Rendition, Stylized-ImageNet, ImageNet-C | accuracy / mCE: 44.77 -> 41.06; 19.38 -> 17.03; 46.22 -> 59.88 | Hurt |
| Replace concatenation with cross-attention (ablation) | Ours ViT-B · ImageNet-Rendition, Stylized-ImageNet, ImageNet-C | accuracy / mCE: 44.77 -> 43.75; 19.38 -> 18.67; 46.22 -> 50.06 | Hurt |
| Concatenate continuous global CNN features instead of discrete embeddings (ablation) | ViT-B · ImageNet robustness benchmarks | robustness | No clear effect |
| Use joint learning rather than the staged pretraining-finetuning strategy (ablation) | Dr. ViT · ImageNet classification | performance | Hurt |
| Pretrain the codebook on ImageNet-21K rather than ImageNet (ablation) | ViT models trained on ImageNet-21K · ImageNet-Rendition, Stylized-ImageNet, ImageNet-Sketch, ImageNet-C | accuracy / mCE: 51.56 -> 55.26; 14.77 -> 19.69; 41.10 -> 44.72; 55.31 -> 45.86 | Helped |
Closest papers by meaning
- Vision Transformers Are Robust Learners (AAAI 2022 · similarity 0.93)
- Towards Robust Vision Transformers: Path Dependency Analysis and a Simple Two-Stage Adversarial Training (CVPR 2026 · similarity 0.91)
- Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words (ICLR 2024 · similarity 0.89)
- Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.88)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.88)
- Random Entangled Tokens for Adversarially Robust Vision Transformer (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.