Research map / Vision-language and multimodal / Human and embodied modeling
Pedestrian Attribute Recognition: A New Benchmark Dataset and a Large Language Model Augmented Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the MSP60K cross-domain pedestrian attribute dataset (60,122 images, 57 attributes, 8 scenes) with a benchmark of 17 PAR methods, and proposes LLM-PAR, which adds an LLM captioning branch and MEQ-Former part-aware queries to a ViT-based multi-label classifier.
Problem
Existing pedestrian attribute recognition datasets are near saturation, small, use only random splits that ignore domain shift, and lack realistic degradation, so they do not reflect real-world robustness.
Approach
Combine a new multi-scene dataset with cross-domain splits and synthetic degradations, and augment a ViT multi-label classifier with a part-aware multi-embedding query Transformer and an LLM branch that generates attribute descriptions whose outputs are ensembled with classification logits.
Key results
- LLM-PAR reaches 92.25 mA and 90.39 F1 on PETA, versus 89.91 mA for the prior best (OAGCN) on PETA's reported rows.
- On MSP60K cross-domain split, LLM-PAR achieves 66.29 mA and 72.05 F1, the best on nearly all metrics versus PromptPAR at 63.24 mA.
- On PA100K, LLM-PAR reports 91.09 mA and 90.41 F1, exceeding FRDL and PARFormer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM-PAR instead of PromptPAR | PAR models · MSP60K random split | mA: 78.81 -> 80.13 | Helped |
| Use LLM-PAR instead of PromptPAR | PAR models · MSP60K cross-domain split | mA: 63.24 -> 66.29 | Helped |
| Use LLM-PAR instead of PARformer | PAR models · PETA | mA: 89.32 -> 92.25 | Helped |
| Use LLM-PAR instead of FRDL | PAR models · PA100K | mA: 89.44 -> 91.09 | Helped |
| Add LoRA fine-tuning to the visual branch (ablation) | LLM-PAR · PETA | mA: 82.89 -> 90.14 | Helped |
| Add the CLS-LLM branch (ablation) | LLM-PAR visual branch · PETA | mA: 90.14 -> 90.89 | Helped |
| Add AGFA (ablation) | LLM-PAR with CLS-LLM · PETA | mA: 90.89 -> 91.78 | Helped |
| Add CLS-IN (ablation) | LLM-PAR · PETA | mA: +0.22 | Helped |
| Mask ground truth at a 50% rate rather than use ground truth directly (ablation) | LLM-PAR language branch · PETA | mA: 77.03 -> 88.20 | Helped |
| Use 100% masking with padding (ablation) | LLM-PAR language branch · PETA | mA: 88.20 -> 64.59 | Hurt |
| Replace ground truth with random training sentences (ablation) | LLM-PAR language branch · PETA | mA: 88.20 -> 88.84 | Helped |
| Increase AGFA layers from 3 to 12 (ablation) | LLM-PAR · PETA | mA: 92.25 -> 92.77 | Helped |
Surprising findings
- Zero-shot MiniGPT-4 captioning reaches only 56.93 mA on the MSP60K random split, far below trained PAR models.
- Many methods that perform well on random split drop sharply on the cross-domain split (e.g., HAP mA drops by 18.22).
Limitations
- The authors note training and inference require substantial computational resources.
- The dataset is planned for expansion, indicating current scale limits.
- Inconsistent numbers across the paper (e.g., 60,015 vs 60,122 images; PETA ablation values) weaken result reporting.
Experimental setup
Datasets: MSP60K PETA PA100K RAPv1
Benchmarks: MSP60K PETA PA100K RAPv1
Metrics: mA Accuracy Precision Recall F1
Baselines: DeepMAR Strong Baseline RethinkingPAR VTB Label2Label DFDT PARFormer PromptPAR HAP MiniGPT-4 zero-shot
Builds on: MiniGPT-4 BLIP-2 Vicuna CBAM EVA-CLIP
Closest papers by meaning
- Selective and Orthogonal Feature Activation for Pedestrian Attribute Recognition (AAAI 2024 · similarity 0.93)
- Enhanced Visual-Semantic Interaction with Tailored Prompts for Pedestrian Attribute Recognition (CVPR 2025 · similarity 0.89)
- Pedestrian Attribute Recognition as Label-balanced Multi-label Learning (ICML 2024 · similarity 0.88)
- Attribute-Guided Pedestrian Retrieval: Bridging Person Re-ID with Internal Attribute Variability (CVPR 2024 · similarity 0.85)
- Multi-View Pedestrian Occupancy Prediction with a Novel Synthetic Dataset (AAAI 2025 · similarity 0.83)
- Multispectral Pedestrian Detection with Sparsely Annotated Label (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.