Research map / Architectures and efficiency / Attention and transformers
Inverted Pyramid Multi-task Transformer for Dense Scene Understanding
Hanrong Ye, Dan Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add UP-Transformer block (UTB) (ablation) | InvPT Baseline (MT) · NYUD-v2 Semseg | mIoU: 41.06 -> 43.18 | Improved |
| add UP-Transformer block (UTB) (ablation) | InvPT Baseline (MT) · NYUD-v2 Depth | RMSE: 0.6350 -> 0.5643 | Improved |
| add UP-Transformer block (UTB) (ablation) | InvPT Baseline (MT) · NYUD-v2 Boundary | odsF: 76.00 -> 76.10 | No clear effect |
| add cross-scale attention message passing (AMP) (ablation) | InvPT w/ UTB · NYUD-v2 Semseg | mIoU: 43.18 -> 43.64 | Improved |
| add cross-scale attention message passing (AMP) (ablation) | InvPT w/ UTB · PASCAL-Context Semseg | mIoU: 72.34 -> 73.29 | Improved |
| add encoder feature aggregation (EFA) (ablation) | InvPT w/ UTB+AMP · NYUD-v2 Semseg | mIoU: 43.64 -> 44.27 | Improved |
| add encoder feature aggregation (EFA) (ablation) | InvPT w/ UTB+AMP · PASCAL-Context Parsing | mIoU: 61.78 -> 62.73 | Improved |
| add encoder feature aggregation (EFA) (ablation) | InvPT w/ UTB+AMP · NYUD-v2 Boundary | odsF: 76.10 -> 76.10 | No clear effect |
| train all tasks jointly with a plain multi-task decoder instead of single-task | InvPT Baseline · NYUD-v2 Semseg | mIoU: 43.29 -> 41.06 | Hurt |
| use full InvPT multi-task model instead of single-task baseline | InvPT · NYUD-v2 | MT performance Delta_m: +2.59 | Improved |
| use full InvPT multi-task model instead of single-task baseline | InvPT · PASCAL-Context | MT performance Delta_m: +1.76 | Improved |
| scale encoder from Swin-T to Swin-B | InvPT · NYUD-v2 Semseg | mIoU: 44.27 -> 50.97 | Improved |
Closest papers by meaning
- InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding (TPAMI 2024 · similarity 0.95)
- VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · similarity 0.87)
- Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner (NeurIPS 2024 · similarity 0.86)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.86)
- MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding (CVPR 2024 · similarity 0.86)
- ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
