Research map / Architectures and efficiency / Attention and transformers
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use nested local transformers with the proposed block aggregation | NesT · CIFAR10 test | accuracy: 95.97 -> 97.20 | Helped |
| Use nested local transformers with the proposed block aggregation | NesT · CIFAR100 test | accuracy: 80.75 -> 82.56 | Helped |
| Use NesT-B instead of Swin-B | ImageNet-1k · ImageNet-1k | Top-1 accuracy: 83.3 -> 83.8 | Helped |
| Use NesT-S instead of Swin-B | ImageNet-1k · ImageNet-1k | Top-1 accuracy: 83.3 -> 83.3 | No clear effect |
| Use NesT-B instead of Swin-B | ImageNet-1k with ImageNet-22K pre-training · ImageNet-1k | ImageNet accuracy: 86.0 -> 86.2 | Helped |
| Add teacher distillation (ablation) | NesT-B · CIFAR10 | accuracy: 97.2 -> 97.1 | No clear effect |
| Add teacher distillation (ablation) | NesT-B · CIFAR100 | accuracy: 82.6 -> 84.5 | Helped |
| Use a strongly augmented teacher instead of standard augmentation for teacher training (ablation) | NesT-B distilled from PN-164-48 · CIFAR100 | accuracy: 84.5 -> 83.7 | Hurt |
| Use a strongly augmented teacher instead of standard augmentation for teacher training (ablation) | NesT-B distilled from PN-164-270 · CIFAR100 | accuracy: 84.9 -> 83.8 | Hurt |
| Vary MSA head count from 1 to 96 while holding hidden dimension fixed (ablation) | NesT₄-B · CIFAR10 | accuracy: 97.1; 96.85; 96.92; 97.07; 97.21; 97.01; 97.03; 97.08 | No clear effect |
| Use a deeper hierarchy rather than a shallower one (ablation) | NesT · CIFAR | accuracy | Helped |
| Increase hierarchy depth to 5 (ablation) | NesT · CIFAR | accuracy | Hurt |
Closest papers by meaning
- An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.87)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.87)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.87)
- Slicing Vision Transformer for Flexible Inference (NeurIPS 2024 · similarity 0.86)
- Linearly Decomposing and Recomposing Vision Transformers for Diverse-Scale Models (NeurIPS 2024 · similarity 0.86)
- Masked Generative Nested Transformers with Decode Time Scaling (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.