Multivariate calibration of classifier scores into probability space

This book supplies a unifying framework for the derivation of probabilistic membership values in any classification task. While statistical classifiers usually generate such probabilities which reflect the assessment uncertainty, regularization methods supply membership values which do not induce any probabilistic confidence. It is desirable, to transform or re-scale membership values to membership probabilities, since they are comparable and easier applicable for post-processing. In this book several univariate calibration methods are presented. The methods are compared by their performance in experiments measured in terms of correctness and well-calibration. Multivariate extensions for regularization methods usually use a reduction to binary tasks, followed by univariate calibration and further application of the pairwise coupling algorithm. This book introduces a well-performing alternative to coupling that generates Dirichlet distributed membership probabilities. This flexible one-step algorithm bases on probability theory and is applicable to all classification problems. Dirichlet calibration method and pairwise coupling are compared in further experiments.