AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models

Krishna Balasubramanian, Aleksandr Podkopaev, Shiva Kasiviswanathan

ICML 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
model judge dependence with Ising models instead of conditional independence (weighted majority vote)LLM-as-a-judge label aggregation · Relevance (WikiQA)Test accuracy: 0.820 -> 0.912Improved
model judge dependence with Ising models instead of CI weighted majority voteLLM-as-a-judge label aggregation · Toxicity (Jigsaw)Test accuracy: 0.694 -> 0.792Improved
model judge dependence with Ising models instead of CI weighted majority voteLLM-as-a-judge label aggregation · Summarization (CNN/DailyMail)Test accuracy: 0.737 -> 0.801Improved
use class-dependent couplings instead of class-independent couplings (ablation)Ising aggregator · Relevance (WikiQA)Test accuracy: 0.899 -> 0.912Improved
use class-dependent couplings instead of class-independent couplings (ablation)Ising aggregator · Toxicity (Jigsaw)Test accuracy: 0.780 -> 0.792Improved
use class-dependent couplings instead of class-independent couplings (ablation)Ising aggregator · Summarization (CNN/DailyMail)Test accuracy: 0.806 -> 0.801No clear effect
use weighted instead of uniform majority vote (ablation)CI aggregator · Summarization (CNN/DailyMail)Test accuracy: 0.561 -> 0.737Improved
use weighted instead of uniform majority vote (ablation)CI aggregator · Toxicity (Jigsaw)Test accuracy: 0.695 -> 0.694No clear effect
increase number of judges or training samples (ablation)Ising predictors vs CI-WMV · all three datasetsTest accuracyImproved
ignore dependence (CI predictor) under class-conditional Curie-Weiss modelCI predictor · theory, K to infinityrisk: Bayes risk -> 0 vs CI risk min{pi, 1-pi}Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy