Research map / Large language models / Alignment and preferences

Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization

NeurIPS 2026 · Large language models › Alignment and preferences · approach: Reward model design and preference models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a CP tensor-factorization IRT/Bradley-Terry-style model that fuses cheap autorater labels with a small set of human labels to produce prompt-level and category-level leaderboards with simultaneous confidence intervals.

Problem

Fine-grained (prompt-level or homogeneous-subset) evaluation of generative models needs far more labels than humans can afford, while autoraters are biased and inaccurate at the prompt level.

Approach

Model autorater and human ratings jointly with a low-rank CP tensor of model-prompt-rater capabilities under an ordinal logit link; fit autorater parameters first (representation learning), then fit only human rater embedding and cutoffs on a small human calibration set, with simultaneous confidence intervals for category-level rankings.

Key results

• With 10% of human annotations, the method recovers category-specific rankings and significant prompt-level differences (e.g., Imagen vs Muse on Gecko text rendering and counting).
• Across three benchmarks (30 repeated splits), the proposed method (default and fine-tuned) achieves lower test cross-entropy than Constant, Prompt-specific, and P2L baselines at all human budgets.
• Leave-one-out prediction of held-out models' average scores and win-rate differences correlates strongly with ground truth without any of their human labels, preserving pairwise ordering in most cases.

Surprising findings

• Fine-tuning all parameters on human data helps prediction but invalidates the standard confidence intervals.
• For LMArena, LLaMa-3.3-70b is statistically better than Gemini-2.5-Pro on only ~8% of prompts, with ~24% ties.

Limitations

• Relies on low-rank and ordinal logit modeling assumptions that may fail.
• Side-by-side human templates only identify relative capabilities, limiting cross-prompt comparability.
• Uncertainty estimates do not fully propagate first-stage error and can be optimistic; standard CIs do not apply after fine-tuning.

Experimental setup

Datasets: Gecko2K BigGen Bench LMArena arena-human-preference-140k

Benchmarks: Gecko2K BigGen Bench LMArena

Metrics: test cross-entropy loss simultaneous confidence interval coverage held-out model average score prediction win-rate difference prediction

Baselines: Constant (Bradley-Terry / single-rank) Prompt-specific IRT Prompt-to-Leaderboard (P2L) individual autoraters with logistic regression

Builds on: Item Response Theory Bradley-Terry model CP decomposition Prompt-to-Leaderboard

Closest papers by meaning

• Maximizing Signal in Human-Model Preference Alignment (AAAI 2025 · similarity 0.87)
• Revisiting text-to-image evaluation with Gecko: on metrics, prompts, and human rating (ICLR 2025 · similarity 0.86)
• Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.86)
• Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models (ICLR 2024 · similarity 0.86)
• Human Feedback is not Gold Standard (ICLR 2024 · similarity 0.86)
• Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts (ICLR 2025 · similarity 0.85)