Research map / Large language models / Retrieval and knowledge
Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes perceiving LLM knowledge boundaries via semi-open-ended list questions, using an open-source auxiliary model (LLaMA-2-13B) with pseudo-inverse embedding probability reduction to surface ambiguous answers, verified by RAG-based evaluation vs. self-evaluation of GPT-4.
Problem
Existing knowledge-boundary perception works on close-ended or binary answerable/unanswerable questions, which does not fit semi-open-ended questions with many correct answers, some of which are rare and hard, and black-box LLMs do not expose output probabilities to find low-probability answers.
Approach
Use an open-source auxiliary model and estimate the semantic representation of existing high-probability answer anchor tokens via the Moore-Penrose pseudo-inverse of the output embedding matrix, then subtract that estimated probability mass from the auxiliary model's next-token distribution to force generation of new ambiguous answers; verify them by comparing GPT-4 self-evaluation with RAG-based
Key results
- GPT-4 generated incorrect or unverifiable answers in 82.90% of questions, and 40.15% of its ambiguous answers were unqualified.
- The proposed method reached EM 0.481 and F1 0.587 with AOR 0.326, outperforming Prompt (EM 0.300, AOR 0.490) and MASK (EM 0.470, AOR 0.344) on LLaMA-2-13B.
- GPT-4 made inaccurate self-evaluations on 28.47% of ambiguous answers, and LLaMA-2-13B supplied hidden correct answers making up 75.12% of ambiguous answers judged correct by ground truth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed probability-estimation method instead of direct prompting | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | EM: 0.300 -> 0.481 | Helped |
| use the proposed probability-estimation method instead of direct prompting | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | F1: 0.461 -> 0.587 | Helped |
| use the proposed probability-estimation method instead of direct prompting | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | AOR: 0.490 -> 0.326 | Helped |
| replace MASK with the proposed probability-estimation method | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | EM: 0.470 -> 0.481 | Helped |
| replace MASK with the proposed probability-estimation method | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | F1: 0.587 -> 0.587 | No clear effect |
| replace MASK with the proposed probability-estimation method | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (200 questions) | AOR: 0.344 -> 0.326 | Helped |
| use LLaMA-2-13B instead of LLaMA-2-7B with MASK | MASK ambiguous answer discovery · Ambiguous answer discovery (200 questions) | EM: 0.458 -> 0.470 | Helped |
| add the auxiliary model rather than prompt GPT-4 with existing answers (ablation) | Ambiguous answer discovery · Ambiguous answer discovery (GPT-4 response overlap) | AOR: 0.535 -> 0.326 | Helped |
| include inverse-matrix estimation of near-duplicate tokens (ablation) | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (GPT-4 response overlap) | AOR: 0.344 -> 0.326 | Helped |
| set λ to 80 instead of 60 (ablation) | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (GPT-4 response overlap) | AOR: 0.419 -> 0.326 | Helped |
| set λ to 80 instead of 70 (ablation) | LLaMA-2-13B ambiguous answer discovery · Ambiguous answer discovery (GPT-4 response overlap) | AOR: 0.352 -> 0.326 | Helped |
Surprising findings
- GPT-4 is often unaware of its knowledge boundary: it admits to having listed all answers in only about 7% of questions.
- Increasing GPT-4 sampling temperature produced scrambled text with perplexity above 1000 rather than diverse answers.
Limitations
- Ground-truth labels rely on RAG-based web search and human annotation, which is costly and may be imperfect.
- Evaluation uses entity-level EM/F1 against only discovered ambiguous answers, so full answer-set recall is not measured.
Experimental setup
Datasets: LLM-generated semi-open-ended questions across 32 domains (953 questions)
Benchmarks: semi-open-ended QA dataset constructed in this paper (953 questions, 200-question subset for auxiliary evaluation)
Metrics: EM F1 AOR BLEU-1 BLEU-2 BLEU-3 BLEU-4
Baselines: Prompt MASK Auxiliary Model ablation (GPT-4 with in-context examples) Inverse Matrix ablation
Builds on: MASK (average-initialized probability reduction) chain-of-thought prompting WebGPT
Closest papers by meaning
- RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content (NeurIPS 2024 · similarity 0.88)
- Explore What LLM Does Not Know in Complex Question Answering (AAAI 2025 · similarity 0.88)
- SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.88)
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions (NeurIPS 2025 · similarity 0.88)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.88)
- KScope: A Framework for Characterizing the Knowledge Status of Language Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.