Research map / Large language models / Reasoning and chain of thought
Is There No Such Thing as a Bad Question? H4R: HalluciBot for Ratiocination, Rewriting, Ranking, and Routing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains HalluciBot, an encoder (RoBERTa) classifier that predicts a query's hallucination propensity before generation, using multi-agent Monte Carlo labels from perturbed queries answered by gpt-3.5-turbo, and uses it to guide rewriting, ranking, and routing.
Problem
LLM hallucination is usually analyzed after generation, while the quality of the input query and its propensity to induce hallucination before generation is largely unstudied.
Approach
Estimate a query's hallucination rate without invoking an LLM at inference by distilling Monte Carlo statistics from query perturbations answered by multiple gpt-3.5-turbo agents into an encoder classifier, then use its probabilities as a proxy reward for query rewriting, best-of-N ranking, and scenario routing.
Key results
- HalluciBot RoBERTa-base with calibration and threshold achieves 69.5% test accuracy and 76.0% test F1 for binary hallucination prediction.
- Perturbing queries rather than resampling the same query widens the lower-upper accuracy spread by about 13 points and lowers Fleiss's kappa by 12.5 points.
- Rewriting guided by HalluciBot yields a 30.2% positive class transition from hallucinatory to non-hallucinatory, and best-of-N ranking reaches 95.7% accuracy on multiple choice (abstract claim).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scenario tags to the query (ablation) | RoBERTa-base HalluciBot · HalluciBot binary test | accuracy: 66.1 -> 69.0 | Helped |
| add Platt calibration (ablation) | RoBERTa-base HalluciBot · HalluciBot binary test | accuracy: 66.1 -> 69.5 | Helped |
| tune the probability threshold to τ=0.341 (ablation) | calibrated RoBERTa-base HalluciBot · HalluciBot binary test | F1: 73.6 -> 76.0 | Helped |
| add consensus prediction (ablation) | RoBERTa-base HalluciBot · HalluciBot binary test | accuracy: 66.1 -> 68.7 | Helped |
| use multiple query perturbations instead of repeatedly sampling the original query (ablation) | gpt-3.5-turbo Monte Carlo simulation · HalluciBot Monte Carlo, total | Fleiss's κ: 81.5 -> 69.0 | Hurt |
| use multiple query perturbations instead of repeatedly sampling the original query (ablation) | gpt-3.5-turbo Monte Carlo simulation · HalluciBot Monte Carlo, total | accuracy: 68.0 -> 68.6 | Helped |
| use multiple query perturbations instead of repeatedly sampling the original query (ablation) | gpt-3.5-turbo Monte Carlo simulation · HalluciBot Monte Carlo, Multiple Choice | accuracy: 74.0 -> 76.3 | Helped |
| add consensus information to query rewriting (ablation) | HalluciBot Rewrite · hallucinatory queries | positive class transition: 30.2% -> 31.9% | Helped |
| rank N rewrites with HalluciBot instead of using a single rewrite (ablation) | HalluciBot Rank · hallucinatory queries | positive class transition: +19.5% and +20.4% | Helped |
| switch abstractive queries to the extractive scenario | HalluciBot Route · hallucinatory abstractive test queries | positive class transition: +60.0% | Helped |
| use HalluciBot feedback during rewriting instead of an uninformed rewrite | HalluciBot Rewrite · Multiple Choice | accuracy: +1.4% | Helped |
| use HalluciBot feedback during rewriting | HalluciBot Rewrite · Abstractive | generation similarity: +5.2% | Helped |
Surprising findings
- Naive query rewriting without a feedback signal can convert non-hallucinatory queries into hallucinatory ones (negative class transition).
- Scenario-to-extractive routing gives a +60.0% positive class transition versus 9.7% for rewriting without a scenario change.
Limitations
- Labels depend on automated LLM-generated perturbations and sampled outputs, which may introduce noise.
- Sampling for label generation is computationally expensive during training.
- Evaluated only with gpt-3.5-turbo as the output generator and encoder-based classifiers.
Experimental setup
Datasets: SQuADv2 TruthfulQA SciQ MMLU PIQA BoolQ OpenBookQA MathQA ARC WikiQA HotpotQA TriviaQA
Benchmarks: SQuADv2 TruthfulQA SciQ MMLU PIQA BoolQ OpenBookQA MathQA ARC
Metrics: accuracy F1 precision recall Fleiss kappa agreement spread positive class transition
Baselines: RoBERTa-base BERT-base-cased RoBERTa-large
Builds on: RoBERTa BERT Platt calibration self-consistency
Closest papers by meaning
- ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.88)
- Evaluating language models as risk scores (NeurIPS 2024 · similarity 0.88)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.88)
- B-score: Detecting biases in large language models using response history (ICML 2025 · similarity 0.88)
- FINER: MLLMs Hallucinate under Fine-grained Negative Queries (CVPR 2026 · similarity 0.88)
- Listening to the Wise Few: Query–Key Alignment Unlocks Latent Correct Answers in Large Language Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.