Research map / Vision-language and multimodal / CLIP and zero-shot
DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DM-Adapter, a parameter-efficient CLIP adaptation for text-based person retrieval that places sparse mixture-of-adapters in parallel to MLP layers and uses a domain-aware router with learnable prompts plus a load-balancing loss.
Problem
Full fine-tuning of CLIP for text-based person retrieval is expensive and overfits on small person datasets, while existing PETL methods (CSKT) lack fine-grained feature extraction and suffer from routing imbalance when MoE is used.
Approach
Combine a Top-K sparse mixture of adapters (in parallel with frozen CLIP MLP layers in both vision and language branches) with a domain-aware router that adds learnable domain prompts to the gating logits, plus an auxiliary load-balancing loss.
Key results
- On CUHK-PEDES, DM-Adapter reaches 72.17 R@1 and 64.33 mAP, versus 69.70 R@1 and 62.74 mAP for CSKT, using 16M trainable parameters.
- On RSTPReid, DM-Adapter achieves 60.00 R@1 vs 57.75 for CSKT and 50.55 for CFine, which the authors attribute to overfitting of full fine-tuning.
- Ablation on the average R@1 across three datasets rises from 63.89 (MLP-Adapter) to 64.94 (full DM-Adapter).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Sparse Mixture-of-Adapters instead of a single MLP-Adapter (ablation) | CLIP-based person retrieval · CUHK-PEDES | R@1: 71.00 -> 72.00 | Helped |
| Add Sparse Mixture-of-Adapters instead of a single MLP-Adapter (ablation) | CLIP-based person retrieval · RSTPReid | R@1: 58.55 -> 59.05 | Helped |
| Add Sparse Mixture-of-Adapters instead of a single MLP-Adapter (ablation) | CLIP-based person retrieval · ICFG-PEDES | R@1: 62.13 -> 62.13 | No clear effect |
| Add load-balancing loss to Sparse Mixture-of-Adapters (ablation) | CLIP-based person retrieval · CUHK-PEDES | R@1: 72.00 -> 72.14 | Helped |
| Add load-balancing loss to Sparse Mixture-of-Adapters (ablation) | CLIP-based person retrieval · ICFG-PEDES | R@1: 62.13 -> 62.40 | Helped |
| Add load-balancing loss to Sparse Mixture-of-Adapters (ablation) | CLIP-based person retrieval · RSTPReid | R@1: 59.05 -> 59.40 | Helped |
| Add the Domain-Aware Router to Sparse Mixture-of-Adapters with load-balancing loss (ablation) | CLIP-based person retrieval · CUHK-PEDES | R@1: 72.14 -> 72.17 | Helped |
| Add the Domain-Aware Router to Sparse Mixture-of-Adapters with load-balancing loss (ablation) | CLIP-based person retrieval · ICFG-PEDES | R@1: 62.40 -> 62.64 | Helped |
| Add the Domain-Aware Router to Sparse Mixture-of-Adapters with load-balancing loss (ablation) | CLIP-based person retrieval · RSTPReid | R@1: 59.40 -> 60.00 | Helped |
| Use DM-Adapter instead of CSKT | CLIP-based person retrieval · CUHK-PEDES | R@1: +2.47% | Helped |
| Use DM-Adapter instead of CSKT | CLIP-based person retrieval · ICFG-PEDES | R@1: +2.25% | Helped |
| Use DM-Adapter instead of CSKT | CLIP-based person retrieval · RSTPReid | R@1: 60.00 vs 57.75 | Helped |
Surprising findings
- The full-fine-tuning CFine performs worse than PETL methods on RSTPReid, which the authors attribute to overfitting on the smallest dataset.
- Top-K of 1 degrades performance to a single adapter, and increasing the number of experts beyond 6 reduces performance.
Limitations
- Gains over IRRA on CUHK-PEDES and ICFG-PEDES are small or absent; DM-Adapter is described as comparable rather than superior to IRRA.
- Hyperparameters such as the number of experts and Top-K were chosen empirically on the evaluation benchmarks.
Experimental setup
Datasets: CUHK-PEDES ICFG-PEDES RSTPReid
Benchmarks: CUHK-PEDES ICFG-PEDES RSTPReid
Metrics: R@1 R@5 R@10 mAP memory cost trainable parameters
Baselines: CSKT IRRA CFine IRRA-CLIP SAF IVT LGUR CMPM/C ViTAA SSAN TIPCB AXM-Net
Builds on: CLIP CSKT Adapter SDM loss sparse mixture-of-experts
Closest papers by meaning
- An Empirical Study of CLIP for Text-Based Person Search (AAAI 2024 · similarity 0.87)
- Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.86)
- Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval (NeurIPS 2025 · similarity 0.86)
- Causality-Inspired Invariant Representation Learning for Text-Based Person Retrieval (AAAI 2024 · similarity 0.85)
- Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification (CVPR 2025 · similarity 0.85)
- Mixture of Submodules for Domain Adaptive Person Search (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.