Research map / Large language models / Language, safety and interpretability

Set-LLM: A Permutation-Invariant LLM

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Uncertainty quantification for LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Set-LLM, a decoder-only LLM adaptation (removing causal mask and sequential positions, adding set positional encoding and set attention mask) that is provably permutation-invariant to answer-choice order, and evaluates it on four multiple-choice benchmarks with five base models.

Problem

LLMs are sensitive to the order of options in multiple-choice questions and LLM judges, so answers change under reordering of choices; existing mitigations like majority vote over all permutations incur exponential runtime cost.

Approach

Build permutation invariance into the architecture: replace causal masking with prefix masking, drop sequential positions (NoPE), add set position encodings that number set elements from a shared start position, and add a set attention mask that blocks attention between different elements of the same set, with a proof of set permutation equivariance and invariance.

Key results

• Set-LLM (SetMask+SetPE, Ultra pretraining) on Gemma 2B reaches 85.80 / 65.02 / 80.18 / 76.15 on PIQA / ARC / CSQA / SIQA with identical random and adversarial accuracy, beating the Causal Mask+PE majority vote baseline (83.57 / 59.56 / 78.46 / 75.23).
• Set-LLM is reported to outperform the base model in 20/20 adversarial-order cases and 18/20 random-order cases across five base models, with no drop between evaluation modes.
• Causal Mask+PE finetuned Gemma 2B drops from 55.20% random to 23.72% adversarial accuracy on ARC-Challenge.

Measured effects reported in the paper

Surprising findings

• Removing the causal mask and positions (Prefix Mask+NoPE) yields a bag-of-words model that is permutation-invariant but performs poorly, showing SetPE and SetMask are needed to recover accuracy.
• Additional pretraining on UltraFeedback mainly helps adapted architectures, while not improving the finetuned causal baseline.

Limitations

• Experiments are limited to multiple-choice question answering; generalization to generation or other set-text tasks is not tested.
• Invariance in practice requires full 32-bit precision at evaluation, adding constant runtime overhead.
• Each model is finetuned separately per benchmark with LoRA, so the approach requires task-specific training.

Experimental setup

Datasets: UltraFeedback PIQA ARC-Challenge CommonsenseQA SIQA

Benchmarks: PIQA ARC-Challenge CommonsenseQA SIQA

Metrics: accuracy (random-order average) accuracy (adversarial-order worst case)

Baselines: Causal Mask+PE finetuned Causal Mask+PE majority vote Causal Mask+PE Ultra finetuned Prefix Mask+PE

Builds on: RoPE LoRA prefix masking (T5) Deep Sets majority vote over permutations

Closest papers by meaning

• Order-Independence Without Fine Tuning (NeurIPS 2024 · similarity 0.90)
• SETLEXSEM CHALLENGE: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models (NeurIPS 2024 · similarity 0.88)
• Fool Your (Vision and) Language Model with Embarrassingly Simple Permutations (ICML 2024 · similarity 0.88)
• Large Language Model Cascades with Mixture of Thought Representations for Cost-Efficient Reasoning (ICLR 2024 · similarity 0.87)
• Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only (ICLR 2025 · similarity 0.87)
• Representation Consistency for Accurate and Coherent LLM Answer Aggregation (NeurIPS 2025 · similarity 0.87)