Research map / Vision-language and multimodal / Multimodal LLMs
Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativeness
Qifan Yu, Zhebei Shen, Zhongqi Yue, Yang Wu, Bosheng Qin, Wenqiao Zhang, Yunfei Li, Juncheng Li, Siliang Tang, Yueting Zhuang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add informative value alone over random selection (ablation) | random selection · LLaVA-mix-665k (7.5%) | relative performance: 95.3% -> 98.0% | Improved |
| add unique value alone over random selection (ablation) | random selection · LLaVA-mix-665k (7.5%) | relative performance: 95.3% -> 97.3% | Improved |
| add representative value alone over random selection (ablation) | random selection · LLaVA-mix-665k (7.5%) | relative performance: 95.3% -> 97.5% | Improved |
| combine all three values instead of informative value alone (ablation) | DataTailor · LLaVA-mix-665k (7.5%) | relative performance: 98.0% -> 100.1% | Improved |
| add adaptive collaboration weight (ablation) | DataTailor · LLaVA-mix-665k (7.5%) | relative performance: 98.8% -> 100.1% | Improved |
| add adaptive selection proportion per task (ablation) | DataTailor · LLaVA-mix-665k (7.5%) | relative performance: 97.7% -> 100.1% | Improved |
| add both adaptive components (ablation) | DataTailor · LLaVA-mix-665k (7.5%) | relative performance: 97.2% -> 100.1% | Improved |
| add cross-modal domain clustering (ablation) | DataTailor · LLaVA-mix-665k | MLLM relative performance: 98.1% -> 101.3% | Improved |
| raise clustering threshold lambda from 0.1 to 0.5 (ablation) | DataTailor · LLaVA-mix-665k | MLLM relative performance: 101.3% -> 98.5% | Hurt |
| use only representative value instead of 1:1 uniqueness:representativeness (ablation) | DataTailor · LLaVA-mix-665k | MLLM relative performance: 101.3% -> 99.6% | Hurt |
| use only unique value instead of 1:1 uniqueness:representativeness (ablation) | DataTailor · LLaVA-mix-665k | MLLM relative performance: 101.3% -> 99.5% | Hurt |
| increase selection from 7.5% (50k) to 15% (100k) (ablation) | DataTailor · LLaVA-mix-665k | relative performance: 100.1% -> 101.3% | Improved |
Closest papers by meaning
- Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection (ICLR 2025 · similarity 0.90)
- CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization (NeurIPS 2025 · similarity 0.90)
- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.89)
- Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories (ICML 2026 · similarity 0.88)
- SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning (NeurIPS 2024 · similarity 0.88)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
