Research map / Detection and segmentation / CNN features
PairDETR : Joint Detection and Association of Human Bodies and Faces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends Deformable DETR to jointly detect human bodies and faces and associate them per person via an approximated bipartite matching loss, trained end-to-end without NMS-style post-processing.
Problem
Detecting objects and their pairwise relations (here, body-face association for the same person) typically relies on stand-alone detectors plus non-differentiable post-processing; exact set matching over pairs with missing members is NP-hard.
Approach
Approximates the NP-hard fixed-cost max-flow pair matching by padding missing pair members with relatively estimated nodes (e.g., head annotations or expected faces), reducing it to a standard maximum bipartite matching solved with the Hungarian algorithm over Deformable DETR queries that each predict a body-face pair.
Key results
- On CrowdHuman, PairDETR reaches 42.9 mMR-2 (vs. 52.5 for FPN+BFJ), with AP face 72.6 and AP body 87.17.
- On CityPersons, PairDETR achieves 22.22 reasonable mMR-2 vs. 26.4 for BPJDet-L, with AP face 70.2.
- Ablation shows the approximated matching (Head Approximation) gives 42.9 mMR-2 versus 76.9 for MinCost MaxFlow and 48.67 for Body Approximation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PairDETR for joint body-face detection and association instead of BPJDet-L | PairDETR · CityPersons | mMR-2 reasonable: 26.4 -> 22.22 | Helped |
| Use PairDETR for joint body-face detection and association instead of BPJDet-L | PairDETR · CrowdHuman | mMR-2 all: 50.1 -> 42.9 | Helped |
| Use PairDETR for joint body-face detection and association instead of BPJDet-L | PairDETR · CrowdHuman large boxes >96x96 | mMR-2: 53.81 -> 41.31 | Helped |
| Add association training to Deformable DETR (ablation) | Deformable DETR · CrowdHuman | AP face: 72.2 -> 72.6 | Helped |
| Add association training to Deformable DETR (ablation) | Deformable DETR · CrowdHuman | AP body: 86.5 -> 87.1 | Helped |
| Replace the proposed matching approximation with basic matching (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 45.29 | Hurt |
| Replace the proposed matching approximation with MinCost MaxFlow (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 76.9 | Hurt |
| Replace the proposed matching approximation with body approximation (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 48.67 | Hurt |
| Use body relative points instead of pair-adaptive relative points (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 47.57 | Hurt |
| Use face relative points instead of pair-adaptive relative points (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 43.9 | Hurt |
| Use face and body relative points instead of pair-adaptive relative points (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 45.18 | Hurt |
| Replace head annotations with generated body-relative expected-face annotations (ablation) | PairDETR · CrowdHuman | mMR-2: 42.9 -> 42.8 | No clear effect |
Surprising findings
- Replacing head annotations with generated body-relative expected faces yields almost identical results (mMR-2 42.8 vs 42.9), suggesting head annotations are not required.
Limitations
- Built on Deformable DETR with one relative point per query, making extension to general graph prediction hard.
- Requires extra annotations for invisible parts (head boxes or generated expected faces); without them, limited to other approximations.
- Class count grows exponentially (about 2^n) with the number of object types, and small-object performance lags.
Experimental setup
Datasets: CrowdHuman CityPersons COCO
Benchmarks: CrowdHuman CityPersons
Metrics: AP face AP body mMR-2
Baselines: BFJ BPJDet CrowdDet PETR Deformable DETR
Builds on: DETR Deformable DETR BFJ
Closest papers by meaning
- Efficient Two-Stage Detection of Human-Object Interactions With a Novel Unary-Pairwise Transformer (CVPR 2022 · similarity 0.88)
- Relational Learning for Joint Head and Human Detection (AAAI 2020 · similarity 0.87)
- PBADet: A One-Stage Anchor-Free Approach for Part-Body Association (ICLR 2024 · similarity 0.87)
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.86)
- MatchDet: A Collaborative Framework for Image Matching and Object Detection (AAAI 2024 · similarity 0.86)
- GigaHumanDet: Exploring Full-Body Detection on Gigapixel-Level Images (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.