Research map / Vision-language and multimodal / Human and embodied modeling
Improving Visual Recognition with Hyperbolical Visual Hierarchy Mapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Hi-Mapper, a plug-and-play module that builds a probabilistic (Gaussian/MoG) hierarchy tree, decomposes a pre-trained DNN's feature map into a visual hierarchy, and learns it in hyperbolic (Lorentz) space with a hierarchical contrastive loss to boost image classification and dense prediction.
Problem
Pre-trained DNNs (CNNs and ViTs) represent scene hierarchies poorly: existing hierarchy-aware ViTs use symmetric similarity in Euclidean space, which cannot capture asymmetric parent-child inclusion and distorts hierarchical distances.
Approach
Model hierarchy nodes as Gaussians (parents as mixtures of child Gaussians), decompose the feature map into this tree via transformer decoders, learn parent-child relations with a contrastive loss on Lorentz-model geodesic distances, and encode the hierarchy back into the global CLS feature.
Key results
- Hi-Mapper(DeiT-S) reaches 82.6% top-1 on ImageNet-1K vs 79.8% for DeiT-S, and Hi-Mapper(Swin-T) reaches 83.4% vs 81.2%.
- Hi-Mapper(DeiT-S) improves Mask R-CNN box AP by 8.3 over DeiT-S (reproduced baseline) on COCO.
- On ADE20K, Hi-Mapper(DeiT-S) with UperNet gets 46.3 mIoU vs 43.0 for DeiT-S, and Hi-Mapper(Swin-T) gets 46.8 vs 44.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Hi-Mapper to DeiT-T | DeiT-T · ImageNet-1K | Top-1 accuracy: 75.4 -> 74.8 | Hurt |
| Add Hi-Mapper to ResNet-50 | ResNet-50 · ImageNet-1K | Top-1 accuracy: 76.2 -> 78.2 | Helped |
| Add Hi-Mapper to EfficientNet-B4* | EfficientNet-B4* · ImageNet-1K | Top-1 accuracy: 82.9 -> 84.1 | Helped |
| Add Hi-Mapper to DeiT-S | DeiT-S · ImageNet-1K | Top-1 accuracy: 79.8 -> 82.6 | Helped |
| Add Hi-Mapper to Swin-T | Swin-T · ImageNet-1K | Top-1 accuracy: 81.2 -> 83.4 | Helped |
| Add Hi-Mapper to DeiT-T | DeiT-T · COCO val, Mask R-CNN 1x | APbox: 30.3 -> 37.1 | Helped |
| Add Hi-Mapper to DeiT-T | DeiT-T · COCO val, Mask R-CNN 1x | APmask: 27.1 -> 35.7 | Helped |
| Add Hi-Mapper to DeiT-S | DeiT-S · COCO val, Mask R-CNN 1x | APbox: 36.3 -> 42.6 | Helped |
| Add Hi-Mapper to DeiT-S | DeiT-S · COCO val, Mask R-CNN 1x | APmask: 31.7 -> 38.9 | Helped |
| Add Hi-Mapper to DeiT-S | DeiT-S · ADE20K val | mIoU: 43.0 -> 46.3 | Helped |
| Use hyperbolic rather than Euclidean space for hierarchical contrastive learning (ablation) | Hi-Mapper(DeiT-S) · ImageNet-1K | Top-1 accuracy: 79.2 -> 82.0 | Helped |
| Add KL loss to hyperbolic hierarchical contrastive learning (ablation) | Hi-Mapper(DeiT-S) · ImageNet-1K | Top-1 accuracy: 82.0 -> 82.6 | Helped |
Surprising findings
- Applying the hierarchical contrastive loss in Euclidean space (cosine distance) degrades ImageNet accuracy relative to the baseline, while the hyperbolic version improves it substantially (82.0 vs 79.7 without KL).
Limitations
- Gains on some backbones are modest and some comparisons use reproduced baselines (e.g., DeiT-T/S detection marked as reproduced).
- Larger hierarchy depth L leads to entangled object-level representations; hyperparameters N and L require tuning.
Experimental setup
Datasets: ImageNet-1K COCO 2017 ADE20K
Benchmarks: ImageNet-1K COCO 2017 ADE20K
Metrics: top-1 accuracy box AP mask AP mIoU
Baselines: ResNet-50 EfficientNet-B4 DeiT CrossViT PVT DependencyViT Swin Transformer
Builds on: Hyperbolic image embeddings (Khrulkov et al.) Probabilistic embeddings (Vilnis et al.) Hyperbolic image embedding with hierarchical proxies (Kim et al.) DeiT Swin Transformer
Closest papers by meaning
- Fully Hyperbolic Convolutional Neural Networks for Computer Vision (ICLR 2024 · similarity 0.87)
- HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution (CVPR 2025 · similarity 0.85)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.85)
- Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.85)
- Learning Structured Representations with Hyperbolic Embeddings (NeurIPS 2024 · similarity 0.85)
- Transformer brain encoders explain human high-level visual responses (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.