Research map / Large language models / Language, safety and interpretability
PEARL: Towards Permutation-Resilient LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PEARL, a distributionally robust training framework in which a Sinkhorn-based permutation-proposal network adversarially reorders ICL demonstrations, making LLMs robust to demonstration-order attacks.
Problem
LLMs' in-context learning is highly sensitive to the order of demonstrations, enabling permutation-based attacks that are hard to detect and degrade worst-case performance.
Approach
Frames training as a min-max DRO game over the set of all demonstration permutations; a P-Net (encoder + cross-demonstration layer + Sinkhorn operator with Gumbel sampling and entropy regularization) proposes worst-case permutations while the LLM is trained to minimize loss on them, with only the LLM kept at inference.
Key results
- On Llama3-8B held-out tasks at 4 shots, PEARL raises worst-case ROUGE-L from 30.6 (ERM) to 39.6 (+29.4%) and average from 59.7 to 63.1.
- Permutation attacks succeed on over 80% of samples on TMW/CurDial at delta=50% with exhaustive search on LLaMA-3 (reported in the attack analysis).
- PEARL trained on 5 shots and 512-token sequences gives worst-case gains of roughly 24% to 40% at 8-64 shots and 8k-token contexts on Llama3-8B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with PEARL's adversarial permutation DRO instead of ERM | Llama3-8B · 2-shot held-out tasks | Average ROUGE-L: 57.3 -> 62.9 | Helped |
| Train with PEARL's adversarial permutation DRO instead of ERM | Llama3-8B · 2-shot held-out tasks | Worst-case ROUGE-L: 49.4 -> 56.4 | Helped |
| Train with PEARL's adversarial permutation DRO instead of ERM | Llama3-8B · 3-shot held-out tasks | Worst-case ROUGE-L: 38.3 -> 46.9 | Helped |
| Train with PEARL's adversarial permutation DRO instead of ERM | Llama3-8B · 4-shot held-out tasks | Worst-case ROUGE-L: 30.6 -> 39.6 | Helped |
| Train with PEARL on 5-shot demonstrations and evaluate at higher shot counts | Llama3-8B · 8-shot held-out tasks | Worst-case ROUGE-L: 21.3 -> 29.7 | Helped |
| Train with PEARL on 5-shot demonstrations and evaluate at higher shot counts | Llama3-8B · 16-shot held-out tasks | Worst-case ROUGE-L: 21.3 -> 26.3 | Helped |
| Train with PEARL on 5-shot demonstrations and evaluate at higher shot counts | Llama3-8B · 32-shot held-out tasks | Worst-case ROUGE-L: 19.3 -> 26.4 | Helped |
| Train with PEARL on 5-shot demonstrations and evaluate at higher shot counts | Llama3-8B · 64-shot held-out tasks | Worst-case ROUGE-L: 20.6 -> 28.2 | Helped |
| Vary Sinkhorn iteration count and temperature (ablation) | PEARL · Instruction fine-tuning held-out tasks | Average and worst-case performance | No clear effect |
Surprising findings
- PEARL also improves best-case performance over ERM across all datasets and shot settings.
- Sinkhorn iteration count and temperature have little effect on results.
Limitations
- Worst-case evaluated via exhaustive search only on small shot counts and 100-sample test sets, limiting statistical confidence.
- Evaluated on ROUGE-L for a small set of tasks (4 held-out) from Super-Natural Instructions.
- Training cost roughly doubles due to adversarial P-Net training, and results depend on entropy coefficient choice.
Experimental setup
Datasets: Super-Natural Instructions FLAN v2 synthetic linear-function ICL data
Benchmarks: CommonsenseQA Curiosity Dialogue (CurDial) CoLA Tell Me Why (TMW)
Metrics: ROUGE-L worst-case performance average performance attack success rate normalized squared error
Baselines: ERM ERM+DS ERM+IM InfoAC ERM+CL
Builds on: distributionally robust optimization InfoAC Gumbel-Sinkhorn permutation learning
Closest papers by meaning
- Set-LLM: A Permutation-Invariant LLM (NeurIPS 2025 · similarity 0.87)
- Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs (NeurIPS 2025 · similarity 0.87)
- Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning (ICLR 2024 · similarity 0.87)
- PermLLM: Learnable Channel Permutation for N:M Sparse Large Language Models (NeurIPS 2025 · similarity 0.87)
- A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.87)
- Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.