Loss Functions for Binary Classification and Class Probability Estimation




11 views

Unformatted text preview:

LOSS FUNCTIONS FOR BINARY CLASSIFICATION AND CLASS PROBABILITY ESTIMATION YI SHEN A DISSERTATION IN STATISTICS For the Graduate Group in Managerial Science and Applied Economics Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2005 Supervisor of Dissertation Graduate Group Chairperson TO MY PARENTS ii ABSTRACT LOSS FUNCTIONS FOR BINARY CLASSIFICATION AND CLASS PROBABILITY ESTIMATION YI SHEN SUPERVISOR ANDREAS BUJA What are the natural loss functions for binary class probability estimation This question has a simple answer so called proper scoring rules These loss functions known from subjective probability measure the discrepancy between true probabilities and estimates thereof They comprise all commonly used loss functions log loss squared error loss boosting loss which we derive from boosting s exponential loss and cost weighted misclassification losses We also introduce a larger class of possibly uncalibrated loss functions that can be calibrated with a link function An example is exponential loss which is related to boosting Proper scoring rules are fully characterized by weight functions on class probabilities P Y 1 These weight functions give immediate practical insight into loss functions high mass of points to the class probabilities where the proper scoring rule strives for greatest accuracy For example both log loss and boosting loss have poles near zero and one hence rely on extreme probabilities We show that the freedom of choice among proper scoring rules can be exploited when the two types of misclassification have different costs one can choose proper scoring rules that focus on the cost c of class 0 misclassification by concentrating iii near c We also show that cost weighting uncalibrated loss functions can achieve tailoring Tailoring is often beneficial for classical linear models whereas nonparametric boosting models show fewer benefits We illustrate






Loading Unlocking...
Login

Join to view Loss Functions for Binary Classification and Class Probability Estimation and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Loss Functions for Binary Classification and Class Probability Estimation and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?