Research map / Detection and segmentation / 3D detection and driving
FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibration
Zhijian Huang, Sihao Lin, Guiyu Liu, Mukun Luo, Chaoqiang Ye, Hang Xu, Xiaojun Chang, Xiaodan Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add intra- and inter-gradient calibration to the multi-task baseline | BEVFusion-based multi-task baseline · nuScenes val, det:seg=1:1 | delta_MTL (%): 18.3 -> 5.4 | Improved |
| add inter-gradient calibration (IMTL-G on task gradients) (ablation) | multi-task baseline · nuScenes val, det:seg=1:1 | mIoU (%): 44.0 -> 59.5 | Improved |
| add inter-gradient calibration alone (ablation) | multi-task baseline · nuScenes val, det:seg=1:1 | mAP (%): 59.1 -> 57.1 | Hurt |
| add intra-gradient calibration (balance LiDAR and image gradients) (ablation) | multi-task baseline · nuScenes val, det:seg=1:1 | mAP / mIoU: mAP 59.1 -> 59.5, mIoU 44.0 -> 45.0 | Improved |
| manually raise segmentation loss weight from 5 to 10 (ablation) | multi-task baseline · nuScenes val | mAP / mIoU: mIoU 55.7 -> 57.9, mAP 59.8 -> 59.3 | Mixed |
| add calibration across disjoint annotation settings (ablation) | multi-task baseline · nuScenes Disjoint-balance and Disjoint-normal | delta_MTL (%): 21.3 -> 9.8 (balance); 19.3 -> 9.7 (normal) | Improved |
| apply either calibration alone (ablation) | multi-task baseline · nuScenes val | gamma_task / gamma_modal: gamma_task 19.2 -> 2.5 (inter only) | Improved |
| share backbone for multi-task Fuller instead of separate single-task models | Fuller · nuScenes inference | memory and FPS: 6144MB -> 3103MB; 1.20 -> 2.30 FPS | Improved |
| extend to a third task (foreground segmentation) | Fuller · nuScenes val, 3 tasks | delta_MTL (%): 12.8 -> 9.9 | Improved |
Closest papers by meaning
- M3Net: Multimodal Multi-task Learning for 3D Detection, Segmentation, and Occupancy Prediction in Autonomous Driving (AAAI 2025 · similarity 0.89)
- RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception (ECCV 2024 · similarity 0.88)
- DeepInteraction++: Multi-Modality Interaction for Autonomous Driving (TPAMI 2025 · similarity 0.88)
- Beyond Sparsity: Receptive Field Expansion and Cross-Task Fusion for LiDAR Multi-Task Perception (TPAMI 2026 · similarity 0.87)
- Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion (NeurIPS 2025 · similarity 0.86)
- VeXKD: The Versatile Integration of Cross-Modal Fusion and Knowledge Distillation for 3D Perception (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
