Research map / Detection and segmentation / 3D detection and driving
FFAM: Feature Factorization Activation Map for Explanation of 3D Detectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FFAM, an NMF-based feature factorization activation map with object-specific gradient weighting and voxel upsampling that produces visual explanations for LiDAR 3D detectors such as SECOND and CenterPoint.
Problem
Explaining LiDAR-based 3D object detectors, which are black-box models; prior explanation methods target image models or rely on costly perturbation (OccAM) and do not handle sparse 3D voxel scale alignment or object-specific explanations.
Approach
Apply non-negative matrix factorization to 3D backbone voxel features to get concept activations, aggregate them into a global map, refine it with the feature gradient of a target object's loss, and upsample sparse voxel activations to point level with a Manhattan-distance-neighbor Gaussian-weighted query.
Key results
- On KITTI with SECOND, FFAM reaches Deletion AUC 0.071 (lower is better) versus 0.286 for OccAM and 0.134 for ODAM.
- FFAM attains PG 0.991 and enPG 0.664 on KITTI, above ODAM (0.901 / 0.633) and OccAM (0.946 / 0.023).
- FFAM's VEA is 0.388 overall on Waymo Open, versus 0.234 for ODAM and 0.012 for OccAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FFAM instead of the strongest baseline, ODAM | 3D detector explanations on KITTI (SECOND) · KITTI (all classes) | Deletion AUC: 0.134 -> 0.071 | Helped |
| Use FFAM instead of the strongest baseline, ODAM | 3D detector explanations on KITTI (SECOND) · KITTI (all classes) | Insertion AUC: 0.885 -> 0.907 | Helped |
| Use FFAM instead of the strongest baseline, ODAM | 3D detector explanations on KITTI (SECOND) · KITTI (all classes) | Pointing Game: 0.901 -> 0.991 | Helped |
| Use FFAM instead of the strongest baseline, ODAM | 3D detector explanations on KITTI (SECOND) · KITTI (all classes) | enPG: 0.633 -> 0.664 | Helped |
| Use FFAM instead of the strongest baseline, ODAM | 3D detector explanations on KITTI (SECOND) · KITTI (all classes) | VEA: 0.234 -> 0.388 | Helped |
| Add voxel upsampling to object-specific gradient explanations (ablation) | FFAM on SECOND · KITTI | VEA: 0.252 -> 0.335 | Helped |
| Add voxel upsampling to object-specific gradient explanations (ablation) | FFAM on SECOND · KITTI | Pointing Game: 0.933 -> 0.957 | Helped |
| Add feature factorization to object-specific gradient plus voxel upsampling (ablation) | FFAM on SECOND · KITTI | Pointing Game: 0.957 -> 0.991 | Helped |
| Add feature factorization to object-specific gradient plus voxel upsampling (ablation) | FFAM on SECOND · KITTI | VEA: 0.335 -> 0.391 | Helped |
| Add voxel upsampling and feature factorization (ablation) | FFAM on SECOND · KITTI | Insertion AUC: 0.909 -> 0.907 | No clear effect |
| Vary feature-map layer from conv1 through conv4 (ablation) | FFAM on SECOND · KITTI | VEA: 0.084; 0.116; 0.161; 0.208 | Mixed |
| Vary voxel upsampling range and neighbor count (ablation) | FFAM on SECOND · KITTI | VEA: 0.161; 0.218; 0.313; 0.309 | Mixed |
Surprising findings
- Voxel upsampling and feature factorization do not improve the Insertion metric, which the authors attribute to detectors relying on neighbor context.
- Grad-CAM performs poorly on PG and enPG, suggesting classification-based explanation alone is insufficient for detectors.
Limitations
- FFAM requires access to intermediate feature maps inside the 3D detector, so it is not applicable to black-box detectors.
- Hyperparameters (layer choice, NMF rank r, neighbor range and k) were tuned on the evaluation setting, with results reported for a single detector per dataset in main text.
Experimental setup
Datasets: KITTI Waymo Open
Benchmarks: KITTI Waymo Open
Metrics: Deletion Insertion visual explanation accuracy (VEA) Pointing game (PG) energy-based Pointing game (enPG)
Baselines: Grad-CAM ODAM OccAM
Builds on: DFF OccAM ODAM Grad-CAM
Closest papers by meaning
- Fully Sparse 3D Object Detection (NeurIPS 2022 · similarity 0.87)
- ODAM: Gradient-based Instance-Specific Visual Explanations for Object Detection (ICLR 2023 · similarity 0.87)
- Exploring Active 3D Object Detection from a Generalization Perspective (ICLR 2023 · similarity 0.87)
- LinK: Linear Kernel for LiDAR-Based 3D Perception (CVPR 2023 · similarity 0.86)
- iDet3D: Towards Efficient Interactive Object Detection for LiDAR Point Clouds (AAAI 2024 · similarity 0.85)
- STONE: A Submodular Optimization Framework for Active 3D Object Detection (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.