Research map / Large language models / Alignment and preferences
OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment
Liang Lin, Zhihao Xu, Junhao Dong, Jian Zhao, Yuchen Yuan, Guibin Zhang, Miao Yu, Yiming Zhang, Zhengtao Yao, Huahui Yi, HAICHUAN TANG, Dongrui Liu and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| orthogonal subspace updates (OrthAlign) instead of strongest baseline on 3-objective alignment | OrthAlign · HelpSteer2 / UltraFeedback, Llama3-SFT | average score: 70.13 -> 75.95 (HelpSteer2 vs Knots) | Improved |
| OrthAlign vs RSDPO on helpful win rate | OrthAlign · UltraFeedback Llama3-SFT | helpful win rate: 71.92 -> 71.57 | No clear effect |
| OrthAlign vs DPO on truthfulness | OrthAlign · UltraFeedback Llama3-SFT | TruthfulQA MC2: 67.03 -> 66.58 | No clear effect |
| add OrthAlign subspace projection to DPO (ablation) | DPO · HelpSteer2+SafeRLHF | harmless rate: 71.24 -> 93.84 | Improved |
| add OrthAlign subspace projection to MODPO (ablation) | MODPO · HelpSteer2+SafeRLHF | harmless rate: 48.46 -> 79.32 | Improved |
| add OrthAlign subspace projection to SPO (ablation) | SPO · HelpSteer2+SafeRLHF | helpful win rate: 61.24 -> 67.28 | Improved |
| increase fixed subspace rank from 12 to 26 (ablation) | OrthAlign · Helpsteer2+SafeRLHF, Llama3-SFT | harmless rate: 93.80 -> 81.34 | Hurt |
| vary subspace rank (14 onward) (ablation) | OrthAlign · Helpsteer2+SafeRLHF, Llama3-SFT | helpful win rate: 63.59 ~ 65.79 | No clear effect |
| reverse training order to Helpful->Harmless (ablation) | OrthAlign · UltraFeedback+SafeRLHF | average score: 79.59 -> 77.27 | Mixed |
Closest papers by meaning
- Panacea: Pareto Alignment via Preference Adaptation for LLMs (NeurIPS 2024 · similarity 0.91)
- ComPO: Preference Alignment via Comparison Oracles (NeurIPS 2025 · similarity 0.90)
- MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models (NeurIPS 2026 · similarity 0.89)
- Robust LLM Alignment via Distributionally Robust Direct Preference Optimization (NeurIPS 2025 · similarity 0.89)
- Less is More: Improving LLM Alignment via Preference Data Selection (NeurIPS 2025 · similarity 0.89)
- MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
