Research map / Large language models / Alignment and preferences
The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives
Matthieu Bou, Nyal Patel, Arjun Jagota, Satyapriya Krishna, Sonali Parbhoo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sequential Bayesian updates instead of single round (ablation) | Alignment Auditor (Llama-3.2-1B) · toxicity audit | posterior log-det covariance: -196 -> -897 | Improved |
| use sequential Bayesian updates instead of single round (ablation) | Alignment Auditor (Llama-3.2-1B) · toxicity audit | epistemic uncertainty (MI): 0.1272 -> 0.0683 | Improved |
| use sequential Bayesian updates instead of single round (ablation) | Alignment Auditor (Llama-3.2-1B) · toxicity audit | single-text AUROC: 0.7954 -> 0.8323 | Improved |
| use sequential Bayesian updates instead of single round (ablation) | Alignment Auditor (Llama-3.2-1B) · toxicity audit | pairwise accuracy: 0.7597 -> 0.7524 | No clear effect |
| scale base model from Llama-3.2-1B to Llama-3.1-8B | Alignment Auditor · toxicity audit | single-text AUROC: 0.832 -> 0.916 | Improved |
| scale base model from Llama-3.2-1B to Llama-3.1-8B | Alignment Auditor · toxicity audit | Cohen's d: 1.325 -> 1.821 | Improved |
| scale base model from Llama-3.2-1B to Llama-3.1-8B | Alignment Auditor · toxicity audit | pairwise ECE: 0.0425 -> 0.0462 | Hurt |
| scale base model from Llama-3.2-1B to Llama-3.1-8B | Alignment Auditor · HH-RLHF helpfulness | pairwise accuracy: 0.725 -> 0.729 | Improved |
| scale base model from Llama-3.2-1B to Llama-3.1-8B | Alignment Auditor · HH-RLHF helpfulness | pairwise ECE: 0.0328 -> 0.0438 | Hurt |
| use full-posterior reward variance instead of output toxicity probability for spurious-marker detection | Alignment Auditor · Spurious-marker detection | AUC: 0.58 -> 0.88 | Improved |
| use posterior-mean reward instead of output toxicity probability | Alignment Auditor · Spurious-marker detection | AUC: 0.58 -> 0.70 | Improved |
| use round 1 posterior instead of contracted posterior (rounds 2-5) as RLHF reward (ablation) | Alignment Auditor · toxicity PPO alignment | reward hacking | Hurt |
Closest papers by meaning
- Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment (AAAI 2025 · similarity 0.91)
- Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards (ICML 2026 · similarity 0.90)
- Taming Overconfidence in LLMs: Reward Calibration in RLHF (ICLR 2025 · similarity 0.90)
- Post-hoc Reward Calibration: A Case Study on Length Bias (ICLR 2025 · similarity 0.89)
- BLEUBERI: BLEU is a surprisingly effective reward for instruction following (NeurIPS 2025 · similarity 0.89)
- Reward-Augmented Data Enhances Direct Preference Alignment of LLMs (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
