Research map / Large language models / Alignment and preferences
The Realignment Problem: When Right becomes Wrong in LLMs
Aakash Sen Sharma, Debdeep Sanyal, Manodeep Ray, Vivek Srivastava, Shirish Karande, Murari Mandal
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add impact-score weighting instead of uniform weights (ablation) | TRACE · SynthValueBench (Qwen2.5-7B) | target policy agreement (%): 62.8 -> 70.2 | Improved |
| add impact-score weighting instead of uniform weights (ablation) | TRACE · SynthValueBench (Qwen2.5-7B) | GPQA: 28.4 -> 30.1 | Improved |
| add impact-score weighting instead of uniform weights (ablation) | TRACE · SynthValueBench (Qwen2.5-7B) | HellaSwag: 75.1 -> 77.3 | Improved |
| add the triage stage (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | target policy agreement (%): 58.1 -> 70.7 | Improved |
| add impact weighting (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | attack success rate (%): 32.1 -> 27.3 | Improved |
| add impact weighting (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | target policy agreement (%): 62.8 -> 70.7 | Improved |
| add KL regularization (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | MMLU: 64.1 -> 70.2 | Improved |
| add KL regularization (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | target policy agreement (%): 71.5 -> 70.7 | Hurt |
| remove triage stage (ablation) | TRACE · PKU-SafeRLHF (Llama-3.1-8B) | attack success rate (%): 24.6 -> 27.3 | Hurt |
| use TRACE instead of naive oracle DPO | Llama-3.1-8B · PKU-SafeRLHF data scale 5k-20k | target policy agreement (%): 35.9 -> 55.9, 52.2 -> 70.7 | Improved |
| flip 20% of oracle triage labels (ablation) | TRACE · policy agreement | policy agreement (%): 70.2 -> 61.5 | Hurt |
| flip 10% of oracle triage labels (ablation) | TRACE · policy agreement | policy agreement (%): 70.2 -> 67.4 | Hurt |
Closest papers by meaning
- On Evaluating LLM Alignment by Evaluating LLMs as Judges (NeurIPS 2025 · similarity 0.92)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.91)
- Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment (NeurIPS 2025 · similarity 0.91)
- Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples (ICML 2025 · similarity 0.91)
- Aligner: Efficient Alignment by Learning to Correct (NeurIPS 2024 · similarity 0.91)
- TODO: Enhancing LLM Alignment with Ternary Preferences (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
