Research map / Label-efficient and robust learning / Recognition and long-tail

What Gets Measured Gets Managed: Sign-aware Recommendation Needs Sign-aware Evaluation

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Recommendation representation learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that graph-based sign-aware recommenders fail to separate liked from disliked items at ranking time, and proposes signed Recall/HR/NDCG metrics that penalize recommending disliked items, plus a proof-of-concept sign auxiliary loss.

Problem

Sign-aware recommender systems that use negative feedback are evaluated with conventional metrics that give zero utility to both disliked and unobserved items, so they cannot detect that the models rank disliked items highly (valence-blindness).

Approach

Introduce a three-valued signed relevance (+1 liked, -gamma disliked, 0 unobserved) and signed versions of Recall, HR and NDCG that reduce to conventional metrics at gamma=0; diagnose valence-blindness with V-AUC and linear probes on embeddings.

Key results

• On Amazon-CDs, SiReN takes the top SNDCG (.0526) while XSimGCL leads conventional NDCG (.0886); on KuaiRand, GFormer (best NDCG .0718) has negative SNDCG (-.0082).
• SIGformer's V-AUC is below 0.5 on Amazon-CDs (.486) and Amazon-Music (.486), while its linear-probe AUC on Amazon-CDs is .561 for embeddings where SiGRec reaches .696 probe AUC with V-AUC .527.
• The proof-of-concept L_sign raises SNDCG of SIGformer by 26% on Amazon-CDs, 46% on Amazon-Music, and 187% on Epinions, with NDCG within 5%; no configuration helps on KuaiRec.

Surprising findings

• Sign-aware methods with the highest conventional NDCG (e.g., SIGformer on Amazon datasets and Epinions) show V-AUC below 0.5, actively ranking disliked items above liked ones.
• Unsigned baselines with near-zero NDCG on KuaiRec obtain large negative SNDCG values (e.g., GFormer -1.931), revealing catastrophic valence failure hidden by conventional metrics.

Limitations

• Analysis limited to graph-based models with inner-product scoring; sequential and LLM-based recommenders not studied.
• The auxiliary loss is a proof-of-concept and can conflict with relevance in dense-negative settings (no valid config on KuaiRec).
• Choice of gamma affects conclusions; default gamma=1 is domain-neutral and not validated against user cost.

Experimental setup

Datasets: Amazon-CDs Amazon-Music Epinions KuaiRand KuaiRec

Benchmarks: Amazon-CDs Amazon-Music Epinions KuaiRand KuaiRec

Metrics: Recall@20 HR@20 NDCG@20 SRecall@20 SHR@20 SNDCG@20 V-AUC Overlap linear probe AUC Neg > Pos

Baselines: SiReN SiGRec NFARec SIGformer Pone-GNN SDCGCL LightGCN GFormer LightGCL XSimGCL

Builds on: BPR NDCG SIGformer LightGCN

Closest papers by meaning

• Sign-Aware Multimodal Graph Recommendation (AAAI 2026 · similarity 0.86)
• Negative Feedback Really Matters: Signed Dual-Channel Graph Contrastive Learning Framework for Recommendation (NeurIPS 2025 · similarity 0.86)
• Unified Minimax Optimization Framework for Propensity Score Estimation in Debiased Recommendation (AAAI 2026 · similarity 0.84)
• More Than What Was Chosen: LLM-based Explainable Recommendation Beyond Noisy User Preferences (ICLR 2026 · similarity 0.83)
• Quantifying the Potential to Escape Filter Bubbles: A Behavior-Aware Measure via Contrastive Simulation (AAAI 2026 · similarity 0.83)
• Multiple Purchase Chains with Negative Transfer Elimination for Multi-Behavior Recommendation (AAAI 2025 · similarity 0.82)