ToolkitError: Evaluation metric 'log_loss' not recognized

User 2568 | 12/3/2015, 11:52:04 PM

The documentation for boostedtreesclassifier evaluate says it support metric='logloss', however I get the error ToolkitError: Evaluation metric 'logloss' not recognized. The supported evaluation metrics are ('auto', 'accuracy', 'confusionmatrix', 'roccurve').

Note: I'm using GraphLab Server Version: 1.6.1 using the standard Graphlab AWS AMI

Reproduce

modelGBT = gl.boostedtrees_classifier.create(train, target='Category', features=['DayOfWeek', 'PdDistrict'])

modelGBT.evaluate(validate, metric='logloss')

Comments

User 954 | 12/5/2015, 12:42:54 AM

Hi Kevin,

docstring for model.evaluate? in GLC 1.6.1 does not say we support log_loss. Could you point us the reference?

Anyway, if you upgrade to the latest version of GLC (1.7.1), 'log_loss' is supported.

Thanks. Emad


User 2568 | 1/6/2016, 3:45:03 AM

My mistake. I'm new to Dato.

I'm using the AWS AMI and did not realise this was not the latest version. I've not upgraded to 1.7.1 and 'log_loss' is supported.

I was using the online documentation https://dato.com/products/create/docs/generated/graphlab.boostedtreesclassifier.BoostedTreesClassifier.evaluate.html?highlight=evaluate#graphlab.boostedtreesclassifier.BoostedTreesClassifier.evaluate

This does not note the version number, so I'll assume in the future is for the latest.


User 940 | 1/6/2016, 6:18:53 PM

Hi @Kevin_McIsaac ,

Yes, the online documentation refers to the latest version. We're currently looking into versioning our documentation.

If you're ever interested in the docs for a particular function, you can type the function followed by '?' in the ipython session as so:

python graphlab.BoostedTreesClassifier.evaluate?

This will always give documentation associated with the installed version.

Cheers! -Piotr