Research map / Optimization / Optimizers
B2P-Corr: Batch-to-Population Gradient Estimators for Non-Decomposable Correlation Losses
Jingquan Yan, Yuwei Miao, Peiran Yu, Junzhou Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes batch-to-population gradient estimators for training with non-decomposable correlation losses (PCC, CCC) under mini-batch stochastic optimization.
Problem
Correlation metrics such as PCC and CCC depend on dataset-level moments, so naive mini-batch training with local statistics gives biased, high-variance gradients.
Approach
Batch-to-population gradient estimators that correct mini-batch correlation gradients toward population-level statistics for non-decomposable correlation losses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace naive mini-batch PCC with B2P-EMA | KVQ · KoNViD-1k (batch size 2) | PCC: 0.865 -> 0.895 | Improved |
| replace naive mini-batch PCC with B2P-EMA | KVQ · KoNViD-1k (batch size 2) | RMSE: 0.303 -> 0.283 | Improved |
| use B2P-EMA instead of the PLC fine-grained correlation loss | KVQ · KoNViD-1k (batch size 2) | PCC: 0.871 -> 0.895 | Improved |
| replace naive mini-batch PCC with B2P-U | KVQ · KoNViD-1k (batch size 8) | PCC: 0.881 -> 0.906 | Improved |
| replace naive PCC term with B2P-U | EGN · 10xProteomic (batch size 8) | PCC: 0.615 -> 0.660 | Improved |
| replace naive PCC term with B2P-U | EGN · 10xProteomic (batch size 32, default) | PCC: 0.629 -> 0.673 | Improved |
| replace naive CCC term with B2P-U | VEATIC baseline · VEATIC valence (batch size 4) | CCC: 0.437 -> 0.467 | Improved |
| replace naive CCC term with B2P-U | VEATIC baseline · VEATIC valence (batch size 20, default) | CCC: 0.606 -> 0.615 | Improved |
| use stop-gradient moment sketches instead of naive mini-batch PCC gradients | Linear predictor, PCC · Synthetic Exp.1 (B=4) | gradient alignment: 0.16 -> 0.43 | Improved |
| add reservoir-based cross-batch U-statistic sketch (B2P-U vs B2P-EMA) (ablation) | B2P-EMA · Synthetic Exp.1 | alignment std: 0.0275 -> 0.0183 | Improved |
| use B2P-U instead of B2P-EMA for mean gradient alignment (ablation) | B2P-EMA · Synthetic Exp.1 | gradient alignment | No clear effect |
| use B2P-EMA or B2P-U at batch size 2 | Linear predictor, PCC · Synthetic Exp.2 (B=2) | DirectionRecovery, TestPCC | Improved |
Experimental setup
Baselines: naive mini-batch correlation loss
Closest papers by meaning
- Romberg-Extrapolated Zeroth-Order Gradient Estimator: Higher-Order Bias Reduction with Preserved Leading Directional Variance (ICML 2026 · similarity 0.84)
- TIC-TAC: A Framework For Improved Covariance Estimation In Deep Heteroscedastic Regression (ICML 2024 · similarity 0.84)
- Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations (ICLR 2025 · similarity 0.83)
- Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression (ICLR 2025 · similarity 0.83)
- The Value of Covariance Matching in Gaussian DDPMs and the Lanczos Sampler (NeurIPS 2026 · similarity 0.83)
- The Bias-Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
