AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Quantitative Evaluation of Confidence Measures in a Machine Learning World

Matteo Poggi, Fabio Tosi, Stefano Mattoccia

ICCV 2017 · Learning theory and algorithms › Prediction and decision losses · approach: Distributionally robust optimization · 79 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use learning-based CCNN instead of best conventional measure APKR11confidence measures with AD-CENSUS · K12AUC: 0.1806 -> 0.1223Improved
use learning-based CCNN instead of APKR11confidence measures with MC-CNN · K12AUC: 0.0566 -> 0.0297Improved
train ENS in regression mode (ENSr) instead of classification (ENSc)ensemble learning confidence · K12 (AD-CENSUS)AUC rank: 7 -> 5Improved
use ENSc rather than ENSr in cost modulationSGM cost modulation · K12/K15/M14bad3 / bad1 error: 10.42 -> 10.635Improved
add confidence-based cost modulation using CCNN to SGMSGM · K12bad3 error rate: 16.53 -> 10.613Improved
add confidence-based cost modulation using CCNN to SGMSGM · M14bad1 error rate: 25.91 -> 22.861Improved
add confidence-based cost modulation using CCNN to SGMSGM · K15bad3 error rate: 13.68 -> 8.791Improved
increase training samples from 5 to 25 K12 stereo pairs for random-forest measures (ablation)ENSr, GCP, LEV, O1 · K12 (AD-CENSUS)Delta_k: less than 0.05 improvementNo clear effect
increase CNN-based measure training data to 2.7 million samples (ablation)CCNN, PBCP · K12 (AD-CENSUS)Delta_k: 0.7 -> 2.7 million samplesImproved
use small training subset for CCNN (ablation)CCNN · K12 (AD-CENSUS)Delta_k: about 0.25 marginHurt
train on M14 instead of K12 for generalization test (ablation)ENSr, GCP, LEV vs CCNN, PBCP, O1 · M14 (AD-CENSUS)Delta_kMixed
use full-patch size variants versus naive (second minimum) variantsconventional confidence measures · K12/K15/M14AUCHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy