Research map / Large language models / Alignment and preferences

LOGO --- Long cOntext aliGnment via efficient preference Optimization

ICML 2025 · Large language models › Alignment and preferences · approach: Active and token-level preference learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LOGO, a reference-free SimPO-style preference optimization with positional indices synthesis, to align long-context models for generation quality on a single 8xA800 node in 16 hours.

Problem

Long-context models can locate salient information but generate misaligned outputs such as hallucinations and instruction-unfollowing; prior data-centric fixes are costly and memory-heavy for long sequences.

Approach

Applies reference-free preference optimization (SimPO-style) to long-context alignment with multiple dis-preference samples built by mixing relevant and irrelevant context chunks, plus an SFT regularizer, and uses positional indices synthesis to simulate longer contexts within GPU memory limits.

Key results

• Llama-3-8B-Instruct-80K improves from 42.3 to 47.0 average on LongBench after LOGO-80K, versus 41.8 and 42.8 for full and partial instruction tuning.
• Llama-3-8B-Instruct-8K scaled to 64K with LOGO reaches 43.9 LongBench average, above YaRN-64K (40.9) and RandPOS-64K (36.8).
• Llama-2-7B-Instruct-80K rises from 29.0 to 36.1 LongBench average with LOGO.

Measured effects reported in the paper

Surprising findings

• Standard CE-based instruction tuning can degrade a well-performing long-context model, e.g. Full instruction tuning lowers Llama-3-8B-Instruct-80K's LongBench average from 42.3 to 41.8.
• Increasing the number of dis-preference samples improves real-world task performance but slightly hurts language modeling.

Limitations

• Hard to identify hallucinations and misaligned outputs automatically; NER-based evaluator is inadequate for error patterns.
• LongBench results are sensitive to differing prompts, hindering direct replication of other works' numbers.
• Memory cost grows with number of dis-preference instances M, limiting scaling.

Experimental setup

Datasets: long-llm-data RedPajama LongAlpaca

Benchmarks: LongBench Needle-in-a-Haystack PG-19 MMLU TruthfulQA ARC

Metrics: average score retrieval score recall score perplexity accuracy

Baselines: YaRN RandPOS Instruct Tuning (Full) Instruct Tuning (Partial) GPT-3.5-Turbo-16K LongChat-v1.5-7B-32k Llama-3.1-8B-Instruct-128K

Builds on: DPO SimPO YaRN LongAlpaca

Closest papers by meaning

• LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions (NeurIPS 2025 · similarity 0.89)
• Long-Short Alignment for Effective Long-Context Modeling in LLMs (ICML 2025 · similarity 0.88)
• A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts (ICLR 2025 · similarity 0.88)
• LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens (ICML 2024 · similarity 0.87)
• LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (ICLR 2025 · similarity 0.87)
• Align-Pro: A Principled Approach to Prompt Optimization for LLM Alignment (AAAI 2025 · similarity 0.86)