Research map / Vision-language and multimodal / Cross-modal learning
FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlexiReID, a CLIP-based dual-encoder with adaptive mixture-of-experts and cross-modal query fusion that supports seven query-to-RGB retrieval modes over text, sketch, and infrared inputs, evaluated on a newly constructed CIRS-PEDES dataset.
Problem
Existing cross-modal person re-identification methods support retrieval only between specific modality pairs (usually non-RGB to RGB), and cannot handle arbitrary combinations of query modalities such as text, sketch, and infrared against RGB galleries.
Approach
An adaptive expert-allocation MoE that activates a variable number of adapter experts per input (by a confidence threshold plus an entropy-based adaptive loss) instead of fixed Top-K, combined with a Cross-Modal Query Fusion module that uses learnable embeddings to fill in missing query modalities and produces seven fused query features matched to RGB via SDM loss.
Key results
- On CUHK-PEDES text-to-RGB, FlexiReID reports R@1 69.20 vs CSKT 69.70 (close to SOTA, not best).
- On the full T+S+IR→R setting FlexiReID reaches 88.23 R@1 on CUHK-PEDES, above its single-modality results.
- Adaptive routing achieves 83.90 average R@1 across seven modes versus 80.58 for Top-K routing in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FlexiReID instead of UNIReID | text-to-RGB retrieval · ICFG-PEDES | R1: 61.28 -> 61.34 | Helped |
| use FlexiReID instead of UNIReID | text-to-RGB retrieval · RSTPReid | R1: 60.25 -> 55.79 | Hurt |
| use FlexiReID instead of UNIReID | sketch-to-RGB retrieval · CUHK-PEDES | R1: 84.87 -> 84.92 | Helped |
| combine text and sketch queries instead of using a sketch query alone | FlexiReID · CUHK-PEDES | R1: 84.92 -> 87.47 | Helped |
| combine text, sketch, and infrared queries | FlexiReID · CUHK-PEDES | R1: 88.23 | Helped |
| add MLP-Adapter to zero-shot CLIP (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 4.49 -> 80.58 | Helped |
| replace MLP-Adapter with AEA-MoE without adaptive loss (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 80.58 -> 82.52 | Helped |
| add adaptive loss to AEA-MoE (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 82.52 -> 83.14 | Helped |
| add CMQF without learnable embedding features (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 83.14 -> 83.73 | Helped |
| add learnable embedding features to CMQF (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 83.73 -> 83.90 | Helped |
| use adaptive routing instead of Top-K routing (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 80.58 -> 83.90 | Helped |
| use adaptive routing instead of soft routing (ablation) | FlexiReID · CUHK-PEDES, seven retrieval modes | Avg. R1: 81.80 -> 83.90 | Helped |
Surprising findings
- On the ICFG-PEDES text-to-RGB task, the reported mINP is very low (7.53) despite strong R@1, and FlexiReID is not best on CUHK-PEDES T→R R@1.
Limitations
- Sketch and infrared modalities for CUHK-PEDES, ICFG-PEDES, RSTPReid are synthetically generated by StyleGAN3/InfraGAN, not real captured data.
- Text-to-RGB and some single-modality results are close to but below SOTA baselines.
- Adaptive loss and threshold hyperparameters are tuned with inconsistent values (0.6 in implementation vs 0.4 in analysis).
Experimental setup
Datasets: CUHK-PEDES ICFG-PEDES RSTPReid SYSU-MM01
Benchmarks: CUHK-PEDES ICFG-PEDES RSTPReid SYSU-MM01
Metrics: Rank-1 Rank-5 Rank-10 mAP mINP
Baselines: UNIReID CFine CSKT DALNet Sketch Trans+ SDCL GUR MPANet
Builds on: CLIP SDM loss MLP-Adapter
Closest papers by meaning
- ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model (NeurIPS 2025 · similarity 0.87)
- Multi-Modal Object Re-identification via Sparse Mixture-of-Experts (ICML 2025 · similarity 0.87)
- Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification (CVPR 2025 · similarity 0.87)
- Implicit Discriminative Knowledge Learning for Visible-Infrared Person Re-Identification (CVPR 2024 · similarity 0.86)
- MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification (NeurIPS 2025 · similarity 0.86)
- Differentiable Auxiliary Learning for Sketch Re-Identification (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.