Research map / Large language models / Reasoning and chain of thought

Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BASIS, a training-free Bayesian method that infers the task-relevant target and similarity criterion from weak test-time feedback (scores, pairwise preferences, verifier outputs) using expected-information-gain probes.

Problem

Language model systems make test-time similarity-based decisions (exemplar retrieval, reranking, response revision), but the task-relevant similarity criterion varies by instance and fixed similarity or static mixtures can fail under criterion shift.

Approach

Maintain a joint posterior over candidate targets and a finite library of relevance criteria, update it from weak feedback with explicit likelihoods (Gaussian, Bradley-Terry, logistic), select probes by expected information gain, and rank candidates by posterior-weighted criterion utility.

Key results

• Six-probe concept-recovery Success@1 rises from 56.0% (tuned static mixture) to 68.1% for BASIS.
• Reasoning accuracy 73.1% for BASIS vs 72.3% for tuned mixture (paired interval [-0.9, 2.5]); refinement win rate 63.8% vs 61.4%.
• Under distractors, reasoning drops 4.9 points for BASIS vs 6.5 for tuned mixture.

Surprising findings

• The reasoning gain over the tuned static mixture (0.8 points) has a confidence interval that includes zero, so it is not significant.
• Latency increases 43-45% over the tuned mixture, and best-of-N verifier selection is comparably accurate on reasoning (72.1%).

Limitations

• Performance depends on the supplied criterion library containing a relevant criterion; cannot discover new criteria.
• Most feedback comes from offline verifiers or rubric scores; live human feedback untested.
• Experiments use a single retriever, generator, and verifier stack.

Experimental setup

Datasets: GSM8K StrategyQA BIG-Bench Hard Anthropic HH-RLHF OpenAssistant OASST1 UltraFeedback WordNet ConceptNet SimLex-999 WordSim-353

Benchmarks: GSM8K StrategyQA BIG-Bench Hard Anthropic HH-RLHF OpenAssistant OASST1 UltraFeedback controlled concept recovery

Metrics: Success@1 accuracy win rate latency attribute match

Baselines: Dense retrieval Tuned static mixture Best-of-N verifier BM25 LLM reranker Rubric-guided LLM reranker Oracle-criterion

Builds on: Bradley-Terry model Self-Refine Information Pursuit

Closest papers by meaning

• Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models (ICLR 2024 · similarity 0.87)
• Inference-time Alignment in Continuous Space (NeurIPS 2025 · similarity 0.87)
• Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment (ICML 2025 · similarity 0.87)
• When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.86)
• Representation Consistency for Accurate and Coherent LLM Answer Aggregation (NeurIPS 2025 · similarity 0.86)
• Interpreting Learned Feedback Patterns in Large Language Models (NeurIPS 2024 · similarity 0.86)