Research map / Large language models / Retrieval and knowledge
POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes POQD, which uses an LLM query decomposer whose prompt is optimized by an LLM-based prompt optimizer alternately with fine-tuning the downstream RAG generator, to improve multi-vector retrieval for RAG QA.
Problem
Query decomposition for multi-vector retrieval strongly affects downstream RAG performance, but the decomposition is non-differentiable and jointly optimizing it with the downstream model is expensive.
Approach
Treat the query-decomposition prompt as the optimized object: an LLM prompt optimizer proposes prompt prefixes scored by downstream training loss, and an alternating algorithm trains the generator for only a few iterations between prompt updates; a theorem links this to loss decrease under PL* and smoothness assumptions.
Key results
- POQD achieves 62.22 text-QA EM on WebQA vs 61.14 for the best baseline ColBERT-orig, and 61.74 vs 49.13 on MultiModalQA image QA vs ColBERT-orig (the paper reports up to 12.61% QA gain).
- Retrieval Hit@1 on image QA improves to 58.26 on MultiModalQA (vs 54.34 for ICL-QD) and 42.33 on WebQA.
- Query decomposition prompt optimization adds negligible training cost relative to generator training, according to the time analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use POQD query decomposition instead of the strongest listed baseline | RAG with multi-vector retrieval · WebQA, MultiModalQA, and ManyModalQA image and text QA | retrieval accuracy and Exact Match accuracy: Hit@1: 41.69 -> 42.33, 54.34 -> 58.26, 28.15 -> 28.67; Hit@2: 52.16 -> 53.24, 79.89 -> 80.58, 87.07 -> 92.35; Exact Matc | Helped |
| use POQD instead of the strongest listed baseline | StrategyQA RAG · StrategyQA | Exact Match accuracy: 73.33 -> 75.55 | Helped |
| use POQD with GPT-4 rather than ICL-QD with GPT-4 for query decomposition (ablation) | WebQA text QA · WebQA | Top-2 retrieval accuracy: 56.83 -> 58.99 | Helped |
| use POQD with DeepSeek-V3 rather than ICL-QD with DeepSeek-V3 for query decomposition (ablation) | WebQA text QA · WebQA | QA accuracy: 58.27 -> 60.43 | Helped |
| set retrieved-document count to 1, 2, or 5 rather than use another decomposition method (ablation) | POQD, WebQA text QA · WebQA | End-to-end QA accuracy: At 1: 59.71 -> 61.15; at 2: 61.14 -> 62.22; at 5: 63.66 -> 64.03 | Helped |
| add filtering of irrelevant generated tokens (ablation) | POQD, WebQA text QA · WebQA | Top-2 retrieval accuracy: 52.51 -> 53.24 | Helped |
| add filtering of irrelevant generated tokens (ablation) | POQD, WebQA text QA · WebQA | End-to-end QA accuracy: 61.87 -> 62.22 | Helped |
| replace Llama3.1-8B with Qwen2.5 as generator (ablation) | POQD, WebQA text QA · WebQA | End-to-end QA accuracy: 62.22; 59.35 | No clear effect |
| replace Sentence-Bert with RoBERTa for retrieval embeddings (ablation) | POQD, WebQA text QA · WebQA | Top-2 retrieval accuracy: 53.24; 43.88 | No clear effect |
| increase τ (ablation) | POQD training algorithm · WebQA text QA | training loss and training time | Mixed |
| set α to 0.05 rather than the default 0.02 (ablation) | POQD training algorithm · WebQA text QA | training loss | Hurt |
| set α to 0.01 rather than the default 0.02 (ablation) | POQD training algorithm · WebQA text QA | convergence speed | Hurt |
Surprising findings
- ColBERT with token-level decomposition performs poorly with the default embedding model (e.g., 11.98 Hit@1 on WebQA image QA), and ColBERT-orig is better than ICL-QD on some text QA datasets.
- Standard LLM-prompted decomposition (ICL-QD) with feedback (ICLF-QD) sometimes underperforms plain ICL-QD on text QA with GPT-4 as decomposer.
Limitations
- Theoretical guarantees rely on strong assumptions (PL* condition, L-smoothness, bounded loss) that are only argued for over-parameterized LLMs fine-tuned with GaLore.
- Hyperparameters alpha, tau, kappa are set by default values tuned on WebQA text QA, with limited sensitivity analysis across datasets.
- Retrieval models for downstream training are kept frozen, so retriever is not co-optimized.
Experimental setup
Datasets: WebQA MultiModalQA ManyModalQA StrategyQA
Benchmarks: WebQA MultiModalQA ManyModalQA StrategyQA
Metrics: Hit@1 Hit@2 exact match accuracy
Baselines: Dense retrieval ColBERT ColBERT-orig S-QD U-QD ICL-QD ICLF-QD Query rewrite
Builds on: ColBERT LLM-based prompt optimizer (OPRO) GaLore OUNS
Closest papers by meaning
- MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encoding (NeurIPS 2024 · similarity 0.86)
- Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.86)
- RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering (NeurIPS 2026 · similarity 0.85)
- PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Prediction (CVPR 2025 · similarity 0.85)
- TriSampler: A Better Negative Sampling Principle for Dense Retrieval (AAAI 2024 · similarity 0.84)
- Benchmarking Large Language Models in Retrieval-Augmented Generation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.