Research map / Label-efficient and robust learning / Recognition and long-tail
QuMAB: Query-based Multi-annotator Behavior Pattern Learning
Liyun Zhang, Zheng Lian, Hong Liu, Takanori Takebe, Shozo Nishii, Yuta Nakashima
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Q-Former with annotator queries over frozen encoder (ablation) | Base encoder + classifiers · AMER | accuracy: 0.28 -> 0.84 | Improved |
| use individual classifiers instead of a unified classifier (ablation) | QuMAB · AMER | accuracy: 0.61 -> 0.84 | Improved |
| use individual classifiers instead of a unified classifier (ablation) | QuMAB · STREET Happiness | accuracy: 0.50 -> 0.63 | Improved |
| add shared self-attention among annotator queries (ablation) | QuMAB · AMER | accuracy: 0.73 -> 0.84 | Improved |
| add shared self-attention among annotator queries (ablation) | QuMAB · STREET Safety | accuracy: 0.46 -> 0.58 | No clear effect |
| model each annotator then aggregate (Post-mv) instead of pooling before modeling (ablation) | QuMAB · AMER consensus | accuracy: 0.43 -> 0.60 | Improved |
| use QuMAB instead of PADL under 40% removed annotations | PADL · AMER | accuracy: 0.58 -> 0.66 | Improved |
| use QuMAB instead of PADL under 40% removed annotations | PADL · STREET Happiness | accuracy: 0.43 -> 0.52 | Improved |
| use QuMAB instead of PADL for average drop under sparsity | PADL · STREET and AMER | accuracy drop: 27.4% -> 20.4% | Improved |
| use QuMAB instead of baselines for consensus prediction | MaDL · AMER | accuracy: 0.57 -> 0.60 | Improved |
| use QuMAB instead of baselines for consensus prediction | PADL · STREET Healthiness | accuracy: 0.50 -> 0.58 | Improved |
| use QuMAB instead of baselines for individual annotator modeling | MaDL · STREET Safety | accuracy: 0.46 -> 0.58 | Improved |
Closest papers by meaning
- A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint) (AAAI 2024 · similarity 0.86)
- Diverging Preferences: When do Annotators Disagree and do Models Know? (ICML 2025 · similarity 0.84)
- Collaborative Refining for Learning from Inaccurate Labels (NeurIPS 2024 · similarity 0.84)
- PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling (NeurIPS 2025 · similarity 0.83)
- ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinking (NeurIPS 2025 · similarity 0.83)
- Automating Data Annotation under Strategic Human Agents: Risks and Potential Solutions (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
