Research map / Architectures and efficiency / RNNs and normalization

Efficient Multitask Dense Predictor via Binarization

CVPR 2024 · Architectures and efficiency › RNNs and normalization · approach: Dataset condensation and flow estimation · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Binarizes a multi-task dense predictor (MTI-Net/InvPT-style, HRNet backbone) into Bi-MTPD variants, adding a variational information bottleneck and feature-based distillation to recover information lost to binarization on NYUD-v2 and PASCAL-Context.

Problem

Multi-task dense prediction models (semantic segmentation, depth, normals, boundaries, etc.) are computationally heavy, and naive network binarization severely degrades information flow in these deep models, causing large accuracy loss.

Approach

Fully binarize a multi-task dense predictor (MTI-Net baseline) including backbone, heads and cross-task multi-modal distillation, then restore information flow with a variational information bottleneck layer in forward pass and feature-level knowledge distillation from a full-precision teacher in backward pass; binarizing only the cross-talk module (Bi-MTPD-C) can even improve accuracy over full p

Key results

• On NYUD-v2, MTI + Bi-MTPD-C reaches 47.71 mIoU vs 45.87 for ATRC (FP) while using about 43% less compute, per the paper's claim.
• Binarizing only the multi-modal distillation module (Bi-MTPD-C) on MTI-Net improves PASCAL-Context semantic segmentation by about 1.3 mIoU over FP MTI-Net (62.98 vs 61.70).
• Fully binarized Bi-MTPD-F reaches 39.20 mIoU on NYUD-v2 and 48.10 on PASCAL, versus 16.20 mIoU for a single binary model (Bi-Single) on NYUD-v2.

Measured effects reported in the paper

Surprising findings

• Binarizing only the cross-task distillation module outperforms its full-precision counterpart on segmentation, which the authors frame as a free-lunch effect from filtering task-irrelevant information.
• Fully binarized multi-task models vastly outperform binarized single-task models (39.20 vs 16.20 mIoU on NYUD-v2), suggesting multi-task cross-talk compensates for information loss.

Limitations

• Fully binarized variant (Bi-MTPD-F) still has a notable accuracy drop relative to full-precision SoTA on several tasks.
• Gains are reported primarily on two datasets (NYUD-v2 and PASCAL-Context) with HRNet-based backbones; generalization to other architectures is only partially shown with InvPT.

Experimental setup

Datasets: NYUD-v2 PASCAL-Context

Benchmarks: NYUD-v2 PASCAL-Context

Metrics: mIoU RMSE mean angle error odsF maxF parameter storage (MB) FLOPs

Baselines: MTI-Net ATRC InvPT PAD-Net ASTMT Cross-stitch PAP

Builds on: MTI-Net Bi-Real Net IR-Net InvPT ATRC

Closest papers by meaning

• Multi-Task Dense Prediction via Mixture of Low-Rank Experts (CVPR 2024 · similarity 0.86)
• Parameter-, Memory-, Time-Efficient Multi-Task Dense Vision Adaptation (AAAI 2026 · similarity 0.86)
• Learning Multiple Dense Prediction Tasks From Partially Annotated Data (CVPR 2022 · similarity 0.85)
• BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons (NeurIPS 2022 · similarity 0.85)
• Layerwise Progressive Freezing: A Training Scaffold for Depth-Scalable Binary Networks (NeurIPS 2026 · similarity 0.83)
• Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner (NeurIPS 2024 · similarity 0.83)