Research map / Large language models / Retrieval and knowledge
ComLQ: Benchmarking Complex Logical Queries in Information Retrieval
Ganlin Xu, Zhitao Yin, Linghao Zhang, Jiaqing Liang, Weijia Lu, Xiaodong Zhang, Zhifei Yang, Sihang Jiang, Deqing Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase query complexity from 1p to 2p to 3p (more hops) | all retrievers · ComLQ | nDCG@10: InteR 71.8 -> 64.4 -> 58.7 (1p, 2p, 3p) | Hurt |
| queries with negation instead of without | all retrievers · ComLQ | nDCG@10: InteR total 55.7; 2in 39.3, 3in 34.7 | Hurt |
| projection-then-intersection ordering (pi) vs intersection-then-projection (ip) | all retrievers · ComLQ | nDCG@10: InteR 52.3 (pi) vs 55.8 (ip) | Hurt |
| require evidence from more supporting passages (#1 to #3) | all retrievers · ComLQ (3p, 3i, 3in, pin) | nDCG@10 | Hurt |
| use sparse BM25 instead of dense BGE or HyDE | retrievers · ComLQ | nDCG@10: BGE 47.4, HyDE 45.6 -> BM25 50.5 | Improved |
| add LLM query rewriting (InteR) on top of BGE embedding | bge-small-en-v1.5 · ComLQ | nDCG@10: 47.4 -> 55.7 | Improved |
| add LLM query rewriting (HyDE) on top of BGE embedding | bge-small-en-v1.5 · ComLQ | nDCG@10: 47.4 -> 45.6 | Hurt |
| LLM query rewriting keeps the negated term in the rewritten query | HyDE, InteR, LameR, AGR · ComLQ negation queries | retrieval of negated passages | Hurt |
| add subgraph indicator to the query-generation prompt (ablation) | GPT-4o query generation · ComLQ construction (14 query types) | structure conformity score (1-4) | Improved |
| more operations in negation queries (inp, pin, pni vs 2in, 3in) | all retrievers · ComLQ | LSNC@100: InteR 38.4 (2in) -> 31.7 (pin) | Hurt |
Closest papers by meaning
- Is Complex Query Answering Really Complex? (ICML 2025 · similarity 0.89)
- BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval (ICLR 2025 · similarity 0.88)
- KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval (ICLR 2024 · similarity 0.88)
- Logical Consistency of Large Language Models in Fact-Checking (ICLR 2025 · similarity 0.88)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.88)
- A Foundation Model for Zero-shot Logical Query Reasoning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
