Research map / Vision-language and multimodal / Cross-modal learning
Enhanced Visual-Semantic Interaction with Tailored Prompts for Pedestrian Attribute Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EVSITP, a CLIP-based bimodal pedestrian attribute recognition framework with image-conditional dual prompts, prompt enhancement/regularization, and bidirectional visual-linguistic interaction, and introduces the Celeb-PAR dataset.
Problem
Pedestrian attribute recognition suffers from large intra-class attribute variability, and existing bimodal PAR methods use static prompt templates and simple concatenation fusion that do not allow active collaboration between visual and linguistic modalities. Existing PAR datasets also cover limite
Approach
Generate image-conditioned learnable prompts via cross-attention with visual features (computed once after the text encoder), fuse fixed-template and learnable prompt features with across-group attention plus a regularization loss, and enable bidirectional visual-linguistic interaction via cross-attention with a shared token, classifying by similarity between the resulting features.
Key results
- On PETA, PA100K, RAPv1, and RAPv2, EVSITP reports mA of 89.65, 88.66, 86.10, and 83.83, improving mA over the second-best PromptPAR by 0.89, 1.19, 0.65, and 0.69 points.
- On the new Celeb-PAR dataset, EVSITP reaches 78.61 mA and 80.40 F1, surpassing PromptPAR by 0.25 points in mA.
- Ablation on RAPv1 shows the full model (fixed prompt, learnable prompt, PERM, BMIM) reaches 86.10 mA versus 82.57 for the baseline without these modules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EVSITP instead of PromptPAR | PAR model · PETA | mA: +0.89% | Helped |
| use EVSITP instead of PromptPAR | PAR model · PA100K | mA: +1.19% | Helped |
| use EVSITP instead of PromptPAR | PAR model · RAPv1 | mA: +0.65% | Helped |
| use EVSITP instead of PromptPAR | PAR model · RAPv2 | mA: +0.69% | Helped |
| add fixed prompt templates (ablation) | EVSITP · RAPv1 | mA: 82.57% to 83.87% | Helped |
| add fixed prompt templates (ablation) | EVSITP · RAPv2 | mA: 79.15% to 81.66% | Helped |
| add learnable prompt tokens (ablation) | EVSITP with fixed prompt · RAPv1 | mA: 83.87% vs. 84.25% | Helped |
| add learnable prompt tokens (ablation) | EVSITP with fixed prompt · RAPv2 | mA: 81.66% vs. 82.35% | Helped |
| add PERM to dual-prompt features (ablation) | EVSITP · RAPv1 | mA | Helped |
| add PERM to dual-prompt features (ablation) | EVSITP · RAPv2 | mA | Helped |
| add BMIM for bidirectional visual-linguistic interaction (ablation) | EVSITP · RAPv1 | mA: +0.91% | Helped |
| add BMIM for bidirectional visual-linguistic interaction (ablation) | EVSITP · RAPv2 | mA: +0.95% | Helped |
Limitations
- Celeb-PAR is annotated from a single long-term re-identification dataset (Celeb-reID), so its scope is tied to that source.
- Gains over PromptPAR are small on several datasets (e.g., about 0.25 mA on Celeb-PAR and 0.65 mA on RAPv1).
Experimental setup
Datasets: PETA PA100K RAPv1 RAPv2 Celeb-reID PETAzs RAPzs
Benchmarks: PETA PA100K RAPv1 RAPv2 Celeb-PAR PETAzs RAPzs
Metrics: mA accuracy precision recall F1 mFive
Baselines: VTB PromptPAR DAFL SOFAFormer AttExpIB-Net Label2Label IAA-Caps SSC
Builds on: CLIP CoCoOp VTB PromptPAR
Closest papers by meaning
- A Pedestrian is Worth One Prompt: Towards Language Guidance Person Re-Identification (CVPR 2024 · similarity 0.90)
- Pedestrian Attribute Recognition: A New Benchmark Dataset and a Large Language Model Augmented Framework (AAAI 2025 · similarity 0.89)
- Selective and Orthogonal Feature Activation for Pedestrian Attribute Recognition (AAAI 2024 · similarity 0.88)
- Attribute-Guided Pedestrian Retrieval: Bridging Person Re-ID with Internal Attribute Variability (CVPR 2024 · similarity 0.84)
- Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification (AAAI 2024 · similarity 0.84)
- DoGA: Enhancing Grounded Object Detection via Grouped Pre-Training with Attributes (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.