Research map / Label-efficient and robust learning / Domain adaptation and generalization

Cloud Object Detector Adaptation by Integrating Different Source Knowledge

NeurIPS 2024 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Geometric alignment and synthetic-to-real adaptation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes COIN, a divide-and-conquer method that adapts a cloud object detector (GDINO) to unlabeled target domains by combining it with a CLIP-based detector, separating detections into consistent, inconsistent and private sets, and fusing inconsistent ones with a gradient-alignment-trained CKG network.

Problem

Adapting a cloud object detector accessible only via API, with no source data or model access and no target labels, to a specific target domain where its detections are still error-prone.

Approach

Integrates a cloud detector with an open CLIP-based Faster R-CNN detector via knowledge separation into consistent, inconsistent, and private detections; inconsistent detections are fused by a Consistent Knowledge Generation network trained to align its gradients with those from consistent detections, with a Mean-Teacher update of the CLIP detector.

Key results

• COIN reaches 39.0% mAP on Foggy-Cityscapes under GDINO, versus 34.4% for the cloud detector alone.
• On KITTI, COIN achieves 80.8 AP for Car, compared with 45.2 for the cloud detector and 62.1 for CLIP.
• On Sim10K, COIN improves over the cloud detector by +15.9 mAP (to 62.4 for car).

Measured effects reported in the paper

Surprising findings

• The CLIP detector alone (38.1% mAP) outperforms the averaged and score-weighted fusion variants on Foggy-Cityscapes ablation, showing the benefit of the Mean-Teacher integration.

Limitations

• Relies on a single public CLIP model and a specific cloud detector (GDINO or GLIP), with performance depending on the cloud detector quality.
• Results reported on a limited set of benchmarks with single-run results and no variance reported.
• Gradient-alignment requires computing gradients through the transformation network for CKG training, adding computation.

Experimental setup

Datasets: Cityscapes Foggy-Cityscapes Clipart BDD100K KITTI Sim10K

Benchmarks: Foggy-Cityscapes BDD100K Clipart KITTI Cityscapes Sim10K

Metrics: mAP@0.5

Baselines: Cloud detector (GDINO) CLIP CLIP detector BiMem LODS IRG SIGMA++ DA-Faster

Builds on: Faster R-CNN CLIP Mean Teacher RegionCLIP Grounding DINO

Closest papers by meaning

• Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection (CVPR 2020 · similarity 0.84)
• Target-Relevant Knowledge Preservation for Multi-Source Domain Adaptive Object Detection (CVPR 2022 · similarity 0.83)
• itKD: Interchange Transfer-Based Knowledge Distillation for 3D Object Detection (CVPR 2023 · similarity 0.83)
• Distilling Image Classifiers in Object Detectors (NeurIPS 2021 · similarity 0.83)
• Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection (CVPR 2023 · similarity 0.83)
• Domain Adaptation for Large-Vocabulary Object Detectors (NeurIPS 2024 · similarity 0.82)