Research map / Label-efficient and robust learning / Recognition and long-tail

Embeddings for Preferences, Not Semantics

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes decorrelated preference tuning (DPT) of sentence encoders on synthetic counterfactual hard triplets, plus a per-topic low-rank projection of frozen embeddings, so that embedding distances track preferential rather than semantic similarity.

Problem

Off-the-shelf text embeddings measure semantic similarity, but facility location and fair clustering over free-form opinions need preferential similarity (endorsement-based distances); cosine's apparent preference signal on natural data is confounded with semantic nuisance.

Approach

Synthesize triplets where the semantic distractor shares the anchor's wording but flips stance and the preference match changes wording but keeps stance, then fine-tune the encoder with a Bradley-Terry loss over cosine margins so cosine down-weights nuisance; additionally learn a rank-r linear projection of frozen embeddings with squared-distance utility per topic.

Key results

• DPT-tuned sentence-T5-XL raises hard-triplet cosine accuracy from 48.3% to 80.0% and natural-triplet mean accuracy from 65.2% to 68.6%.
• Per-topic rank-20 projection of frozen sentence-T5-XL reaches 77.6% natural and 81.1% hard-triplet macro-mean accuracy.
• Base BGE-large, e5-large-v2 and all-mpnet-base score 6.3%, 26.7% and 8.2% on hard triplets; DPT lifts these by roughly 19-52 pp.

Surprising findings

• Cosine accuracy on natural deliberation data (58-60% for several encoders) collapses to 6-27% on hard triplets, suggesting natural-data preference signal is largely driven by correlated nuisance.
• Normal-correlation (NLI-style) training yields only a small lift (59.3% hard) versus hard-triplet training (80.0%), and InfoNCE performs the same as Bradley-Terry.

Limitations

• Surface similarity remains the dominant residual error (61% of sampled errors), and insufficient anchor signal on short Polis comments is not solvable by embeddings.
• Evaluation focuses on within-participant rankings rather than calibrated absolute distances; utility is latent.
• Per-topic projection requires per-topic votes; universal DPT gains on natural data are modest (about +3.4 pp for sentence-T5-XL).

Experimental setup

Datasets: Generative social choice abortion and chatbot surveys Remesh Polarized Issues (campus protests, foreign intervention, right to assemble) Polis conversations (Seattle, Bowling Green, Brexit, Canada, UBI) Habermas Machine dataset Kialo

Benchmarks: 11 online deliberation datasets (GSC, Remesh, Polis) 875 hard triplets

Metrics: triplet accuracy cosine accuracy Wilcoxon signed-rank p-value McNemar test

Baselines: sentence-T5-XL e5-large-v2 BGE-large-en-v1.5 all-mpnet-base-v2 OpenAI text-embedding-3-large Stance-aware SBERT BGE-SparseCL

Builds on: SimCSE Bradley-Terry model LoRA Habermas Machine

Closest papers by meaning

• Limited Preference Data? Learning Better Reward Model with Latent Space Synthesis (NeurIPS 2025 · similarity 0.87)
• Inverse Constitutional AI: Compressing Preferences into Principles (ICLR 2025 · similarity 0.87)
• Anyprefer: An Agentic Framework for Preference Data Synthesis (ICLR 2025 · similarity 0.86)
• Preference Learning for AI Alignment: a Causal Perspective (ICML 2025 · similarity 0.86)
• Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences? (NeurIPS 2025 · similarity 0.86)
• Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes (ICML 2025 · similarity 0.85)