Empty output by model_parameter_search()

User 875 | 4/20/2015, 7:26:22 PM

Hi all,

I have SFrame with 3 columns (userId, trackId, Rating). When I use the modelparametersearch() method, I got the empty output for the column 'precisionrecalloverall' . I use next input:

<pre><code>graphlab.toolkits.modelparametersearch(env, graphlab.recommender.rankingfactorizationrecommender.create, trainfilepath, '/path/data/resultsTest.gl', testfilepath, standardmodelparams = {'userid':'userId','itemid':'trackCode'}, hyperparams={'numfactors':[1,2]}) </code></pre> and another time with target definition:

<pre><code>graphlab.toolkits.modelparametersearch(env, graphlab.recommender.rankingfactorizationrecommender.create, trainfilepath, '/path/data/resultsTest.gl', testfilepath, standardmodelparams = {'target':'Rating', 'userid':'userId','itemid':'trackCode'}, hyperparams={'numfactors':[1,2]}) </code></pre> Output:

<pre><code>In [60]: sf['precisionrecalloverall'] Out[60]: dtype: list Rows: 2 [[{'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}, {'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}, {'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}], [{'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}, {'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}, {'cutoff': 'cutoff', 'precision': 'recall', 'recall': 'precision'}]] </code></pre>

Comments

User 4 | 4/21/2015, 8:02:30 AM

@Guforu Does this still occur with the latest version of GraphLab Create (1.3)? The API examples above are from 1.0 and the implementation has changed considerably since then.


User 875 | 4/21/2015, 8:25:44 AM

No, this is indeed the old version.


User 4 | 4/21/2015, 8:47:28 AM

@Guforu I am not sure about this issue, but since the implementation has changed I suspect it might be fixed in 1.3. Can you try in that release (with the API change)? Thanks!


User 875 | 4/21/2015, 10:10:10 AM

No problem, but currently I can call only the precision and recall for definied length. My output don't have 'precionrecalloverall' anymore. At least I can't find this. I use next code fragment (actually your example from https://dato.com/products/create/docs/graphlab.toolkits.modelparametersearch.html?_ga=1.33684257.1837679012.1421058843):

<pre class="CodeBlock"><code>>>> recommender_data = graphlab.SFrame({'user': range(20), 'item':range(20), 'rating':[1,2,3,4]*5})

job = graphlab.toolkits.modelparametersearch(graphlab.rankingfactorizationrecommender.create, recommenderdata, userid='user', itemid='item', target='rating', numfactors=[1,5]) [INFO] Validating job. [INFO] Validation complete. Job: 'Model-Parameter-Search-Apr-21-2015-11-45-11' ready for execution [INFO] Job: 'Model-Parameter-Search-Apr-21-2015-11-45-11' scheduled. [Class : RankingFactorizationRecommender

Schema

User ID : user Item ID : item Target : rating Additional observation features : 0 Number of user side features : 0 Number of item side features : 0

Statistics

Number of observations : 20 Number of users : 20 Number of items : 20

Training summary

Training time : 0.0847

Settings

ialsconfidencescalingfactor : 1 additionaliterationsifunhealthy: 5 sgdstepadjustmentinterval : 4 numfactors : 1 initrandomsigma : 0.01 maxiterations : 25 regularizationtype : normal sidedatafactorization : 1 regularization : 0.0 unobservedratingvalue : -1.79769313486e+308 numtemperingiterations : 4 sgdstepsize : 0.0 sgdtrialsampleproportion : 0.125 sgdsamplingblocksize : 131072 binarytarget : 0 rankingregularization : 0.25 nmf : 0 ialsconfidencescalingtype : auto trackexactloss : 0 sgdtrialsampleminimumsize : 10000 sgdconvergenceinterval : 4 solver : auto temperingregularizationstartvalue: 0.0 sgdconvergencethreshold : 0.0 sgdmaxtrialiterations : 5 numsamplednegativeexamples : 4 stepsizedecreaserate : 0.75 linearregularization : 0.0 , Class : RankingFactorizationRecommender

Schema

User ID : user Item ID : item Target : rating Additional observation features : 0 Number of user side features : 0 Number of item side features : 0

Statistics

Number of observations : 20 Number of users : 20 Number of items : 20

Training summary

Training time : 0.0672

Settings

ialsconfidencescalingfactor : 1 additionaliterationsifunhealthy: 5 sgdstepadjustmentinterval : 4 numfactors : 5 initrandomsigma : 0.01 maxiterations : 25 regularizationtype : normal sidedatafactorization : 1 regularization : 0.0 unobservedratingvalue : -1.79769313486e+308 numtemperingiterations : 4 sgdstepsize : 0.0 sgdtrialsampleproportion : 0.125 sgdsamplingblocksize : 131072 binarytarget : 0 rankingregularization : 0.25 nmf : 0 ialsconfidencescalingtype : auto trackexactloss : 0 sgdtrialsampleminimumsize : 10000 sgdconvergenceinterval : 4 solver : auto temperingregularizationstartvalue: 0.0 sgdconvergencethreshold : 0.0 sgdmaxtrialiterations : 5 numsampled_negatHTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0


User 4 | 4/21/2015, 5:51:29 PM

Hi @Guforu , I believe this is answered by <a href="http://forum.dato.com/discussion/comment/2962/#Comment_2962"> this comment</a>. Can you confirm?


User 875 | 4/22/2015, 7:22:48 AM

Yes, you unswered my question. Thank you