Research map / Label-efficient and robust learning / Recognition and long-tail
Consistency Training Can Entrench Misalignment
David Demitri Africa, Arathi Mani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply ACT consistency training | misaligned model organism · Reward hacking | misalignment rate change: -55.2% | Improved |
| apply BCT consistency training | misaligned model organism · Reward hacking | misalignment rate change: -48.5% | Improved |
| apply ACT consistency training | misaligned model organism · Sycophancy | misalignment rate change: +18.8% | Hurt |
| apply BCT consistency training | misaligned model organism · Sycophancy | misalignment rate change: +10.0% | Hurt |
| apply ACT/BCT to spurious-correlation organism | misaligned model organism · Spurious correlation | misalignment rate change: +9.5% vs -13.2% | Mixed |
| apply label-generation consistency methods | misaligned model organism · Spurious correlation | sign consistency: 49.7% | No clear effect |
| use k=1 (no selection) instead of best-of-k (ablation) | Self-Rewarding and Self-Confidence labels · Reward hacking, Llama-3.1-8B | misalignment rate change: -22% (Rew), -17% (SC) | Improved |
| replace a fraction of pseudo-labels with 8B-Instruct labels (ablation) | label-generation consistency · Reward hacking, Llama-3.1-8B | suppression | Improved |
| replace a fraction of pseudo-labels with stronger 70B-Instruct labels (ablation) | label-generation consistency · Reward hacking, Llama-3.1-8B | suppression | Hurt |
| add instruction tuning (RLHF) before consistency training (ablation) | consistency training · Sycophancy | misalignment rate change: +19.8% -> -0.2% | Improved |
| use greedy self-training with no selection instead of selection-based consistency (ablation) | label-generation methods · Sycophancy | misalignment rate change: SR +7.8 pp -> -0.7 pp | Improved |
| use greedy self-training with no selection (ablation) | self-generated SFT · Reward hacking | misalignment rate change: -7.1 pp | No clear effect |
Closest papers by meaning
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.86)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.85)
- Aligning Model Properties via Conformal Risk Control (NeurIPS 2024 · similarity 0.85)
- Understanding the Learning Dynamics of Alignment with Human Feedback (ICML 2024 · similarity 0.85)
- Model Spec Midtraining: Improving How Alignment Training Generalizes (NeurIPS 2026 · similarity 0.84)
- Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
