utility for calculating optimal cutoff for classification models

User 2785 | 5/16/2016, 9:34:28 PM

something like what you can find in these R packages: https://cran.r-project.org/web/packages/OptimalCutpoints/OptimalCutpoints.pdf or https://cran.r-project.org/web/packages/pROC/pROC.pdf

Goal is to go from classification label and probability to a optimal cutoff threshold based on Youden or other indexes that take costs of false positives or false negatives into account.

Comments

User 1207 | 5/16/2016, 10:24:13 PM

Hello @wallawalla, that sounds like a great feature request. I'll bring it up with the team here.

One of the things you might find helpful in a larger context is the use of the class_weights parameter for the output class on class training. This can be used to say that errors of one type are worse than another, and internally the algorithms choose the cutoffs based on those weights.

Thanks! -- Hoyt