.compare_models not outputting precision-recall plot

User 2528 | 12/3/2015, 4:17:54 AM

Running Graphlab 1.7.1 and Anaconda Python 2.7.10-4 on Windows 7-64.

Running this code: %matplotlib inline model_performance = graphlab.recommender.util.compare_models(test_data, [popularity_model,personlized_model], user_sample=0.05) should not only compare the models, but produce a Precision-Recall plot showing the curves for both models.

Not getting the plot. Thoughts?

Thanks

Comments

User 19 | 5/3/2016, 6:02:00 PM

Hi,

Please try the following: comparison = gl.compare(validation_data, [model1, model2]) gl.comparison.show_comparison(comparison, [model1, model2])

Please let us know if that doesn't work. We'd be happy to help. Chris


User 954 | 12/5/2015, 12:23:21 AM

Hi, Sorry for the inconvenience.

compare_model plotting capability is hard to discover right now and we will definitely fix it

you can use the following to produce Precision-Recal plot : graphlab.show_comparison(model_performance,[popularity_model,personlized_model]) Regards, Emad


User 3235 | 2/21/2016, 12:36:16 PM

Hello,

getting back to the above mentioned anomaly,

Comparing two models

type(personalized_model)

graphlab.toolkits.recommender.itemsimilarityrecommender.ItemSimilarityRecommender

type(popularity_model)

graphlab.toolkits.recommender.popularity_recommender.PopularityRecommender

graphlab.show_comparison(model_performance,
                     [popularity_model, personalized_model])

TypeError                                 Traceback (most recent call last)
<ipython-input-43-d22bb67d3d5e> in <module>()
  2 
  3 graphlab.show_comparison(model_performance,
----> 4                          [popularity_model, personalized_model])

TypeError: "modelcompsframe" must be a non empty SFrame

Both models are non empty SFrame.

Regards


User 19 | 2/22/2016, 5:37:35 PM

Hi rentze,

The error message suggests that model_performance is empty. Can you let us know what model_performance looks like, and how you produced it?

Cheers, Chris


User 3273 | 3/2/2016, 11:00:22 AM

I use the latest version graphlab 1.8.3 to do the same thing and can not work as above email description.


User 3234 | 3/3/2016, 8:10:05 PM

Same problem : %matplotlib inline model_performance = graphlab.recommender.util.compare_models(test_data, [popularity_model, personalized_model], user_sample=0.05) does not display the expected curve and graphlab.show_comparison(model_performance, [popularity_model, personalized_model]) complains as follows : TypeError: "model_comp_sframe" must be a non empty SFrame


User 15 | 3/8/2016, 12:10:38 AM

@JFM could you show us what the model_performance SFrame looks like, as Chris asked the other user?

Thanks,

Evan


User 3547 | 3/13/2016, 8:36:14 PM

I have the same issue. my model_performance is: ` [{'precisionrecallbyuser': Columns: userid str cutoff int precision float recall float count int

Rows: 52758

Data: +-------------------------------+--------+-----------+--------+-------+ | userid | cutoff | precision | recall | count | +-------------------------------+--------+-----------+--------+-------+ | 00005c6177188f12fb5e2e82cd... | 1 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 2 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 3 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 4 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 5 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 6 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 7 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 8 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 9 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 10 | 0.0 | 0.0 | 1 | +-------------------------------+--------+-----------+--------+-------+ [52758 rows x 5 columns] Note: Only the head of the SFrame is printed. You can use printrows(numrows=m, numcolumns=n) to print more rows and columns., 'precisionrecalloverall': Columns: cutoff int precision float recall float

Rows: 18

Data: +--------+-----------------+------------------+ | cutoff | precision | recall | +--------+-----------------+------------------+ | 1 | 0.0262708973047 | 0.00759800446341 | | 2 | 0.0225179119754 | 0.0124652211014 | | 3 | 0.0209257363812 | 0.0173893380777 | | 4 | 0.0195325827363 | 0.021129046756 | | 5 | 0.018219037871 | 0.0249661210127 | | 6 | 0.0181394290913 | 0.0297444201615 | | 7 | 0.0174002047083 | 0.0331539603162 | | 8 | 0.0162060730126 | 0.0345989925954 | | 9 | 0.0155805754578 | 0.0378640479843 | | 10 | 0.0151142954623 | 0.0408057819394 | +--------+-----------------+------------------+ [18 rows x 3 columns] Note: Only the head of the SFrame is printed. You can use printrows(numrows=m, numcolumns=n) to print more rows and columns.}, {'precisionrecallbyuser': Columns: user_id str cutoff int precision float recall float count int

Rows: 52758

Data: +-------------------------------+--------+-----------+--------+-------+ | userid | cutoff | precision | recall | count | +-------------------------------+--------+-----------+--------+-------+ | 00005c6177188f12fb5e2e82cd... | 1 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 2 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 3 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 4 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 5 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 6 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 7 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 8 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 9 | 0.0 | 0.0 | 1 | | 00005c6177188f12fb5e2e82cd... | 10 | 0.0 | 0.0 | 1 | +-------------------------------+--------+-----------+--------+-------+ [52758 rows x 5 columns] Note: Only the head of the SFrame is printed. You can use printrows(numrows=m, numcolumns=n) to print more rows and columns., 'precisionrecalloverall': Columns: cutoff int precision float recall float

Rows: 18

Data: +--------+-----------------+-----------------+ | cutoff | precision | recall | +--------+-----------------+-----------------+ | 1 | 0.199931763903 | 0.060438HTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0


User 5166 | 5/1/2016, 8:27:06 PM

Hi,

Did anybody found a solution for this? I have exactly the same problem as the rest of the guys above.