Can someone please explain in more detail how to use model_parameter_search method?

User 597 | 8/12/2014, 2:48:21 PM

I am using it for the matrix factorization, but I need to how the object created from this search method is being used to show the recommendation result for particular user/item? Any help of any kind will be appreciated

Comments

User 18 | 8/12/2014, 5:47:53 PM

The API doc provides some information. You can access it through help(func), func?, or online: http://graphlab.com/products/create/docs/generated/graphlab.toolkits.modelparametersearch.html

Please let us know if you have additional questions that are not answered by the above.

A more detailed user guide is also forthcoming.


User 597 | 8/12/2014, 6:41:11 PM

I understand the part of the api, but I don't know what it returns and how it is being used to the model I create from the recommend.create method. Is there an example code that I can look up, or can you write some simple code that uses this method along with the recommend.create method? Thank you.


User 16 | 8/13/2014, 12:36:22 AM

Hi Chan,

I'm working on a more detailed guide right now. Hopefully until that ready this will help.

Once you have your train and test sets in SFrames, you'll need to save them. You'll pass the paths of these SFrames to modelparametersearch.

The first parameter for modelparametersearch is the environment that will be doing the search. This can be EC2 hosts or a Hadoop cluster. Or it could be an environment that executes on your local computer in background. However, it's probably easiest to just have it execute synchronously on your local computer. For that you'll need to create a <a href="http://graphlab.com/products/create/docs/generated/graphlab.deploy.environment.Local.html#graphlab.deploy.environment.Local">Local Environment</a>.

If you're running it locally and synchronously, you don't need to worry about the return value of modelparametersearch. The return values is used to track the progress of the parameter search. If you run it with a local environment, the function will not return until the parameter search is completed.

You also need to pass in the function you want to use to create the model. Sounds like for you this is "graphlab.recommender.create".

The results of the parameter search will be saved (as a SFrame) to the path you give for savepath. You probably don't care about most of the columns in this SFrame. For matrix factorization the most important columns are the tunning parameters (usually just 'nfactors' and 'regularization') and the overall RMSE.

So you're code will probably be very similar to: <pre> import graphlab

testset.save('/data/testset.gl') trainset.save('/data/trainset.gl')

env = graphlab.deploy.environment.Local('local env')

graphlab.toolkits.modelparametersearch( env, graphlab.recommender.create, '/data/trainset.gl', '/data/modelparametersearchresults.gl', '/data/test_set.gl' )

results = graphlab.loadsframe(modelparametersearchresults.gl')

Keep just the columns we care about.

results = results[['nfactors', 'regularization', 'rmseoverall']]

View the parameters with the best RMSE

print results.sort('rmse_overall', ascending = False)

</pre>

Let me know if you have any more questions.

Toby


User 630 | 8/28/2014, 7:08:14 PM

I am testing a model search, but meet a runtime error. Could you help me find what's wrong?

standardmodelparams = {'featurecolumns': newfeatures, 'targetcolumn':'log-casual'} searchspace = { 'params': { 'maxdepth': [10, 15, 20], 'minchildweight': [5, 10, 20] }, 'numiterations': [ntrees] }

job = gl.toolkits.modelparametersearch(env, gl.boostedtrees.create, trainset='/tmp/training', savepath='/tmp/joboutput', testset='/tmp/validation', standardmodelparams=standardmodelparams, hyperparams=search_space )


RuntimeError Traceback (most recent call last) <ipython-input-98-9d008c1c34f6> in <module>() 16 testset='/tmp/validation', 17 standardmodelparams=standardmodelparams, ---> 18 hyperparams=search_space 19 )

/usr/local/lib/python2.7/dist-packages/graphlab/toolkits/modelparametersearch.pyc in modelparametersearch(environment, modelfactory, trainset, savepath, testset, standardmodelparams, hyperparams, maxnummodels, name) 283 pipeline = Pipeline(name + "-Pipeline") 284 pipeline.settasks(tasks) --> 285 return gl.deploy.job.create([pipeline], name=name, environment=environment)

/usr/local/lib/python2.7/dist-packages/graphlab/deploy/job.pyc in create(tasks, name, environment) 179 180 LOGGER.info("Validation complete. Job: '%s' ready for execution" % name) --> 181 job = env.run(session, clonedartifacts, name, environment) 182 _session.register(job) 183 job.save() # save the job once prior to returning.

/usr/local/lib/python2.7/dist-packages/graphlab/deploy/executionenvironment.pyc in run(self, session, tasks, name, environment) 66 """ 67 job = job.Job(name, tasks=tasks, environment=environment) ---> 68 return self.run_job(job, session) 69 70

/usr/local/lib/python2.7/dist-packages/graphlab/deploy/executionenvironment.pyc in runjob(self, job, session) 86 except Exception as e: 87 job._status = 'Failed' ---> 88 raise e 89 90 if session is not None:

RuntimeError: ("Unable to complete task successfully, Exception raised, trace: Traceback (most recent call last):\nAttributeError: 'str' object has no attribute 'items'\n", AttributeError("'str' object has no attribute 'items'",))


User 16 | 8/30/2014, 1:20:57 AM

Hi Wenfeng

Today we just released a new version of GraphLab-Create that supports model parameter search for boosted trees. Previously we didn't support that, which is why your code was not working.

Upgrade to the new version, run: <pre>pip install GraphLab-Create==0.9.1</pre>


User 10 | 9/2/2014, 8:54:16 PM

There is an explanation on using <code class="CodeInline">modelparameterserach</code> in Jay's blog post about using GraphLab Create with the Kaggle competition on bike sharing: http://blog.graphlab.com/using-gradient-boosted-trees-to-predict-bike-sharing-demand. See the section called 'Hyperparameter tuning'.


User 597 | 9/25/2014, 4:33:08 PM

Hi Toby,

Thanks for the comment. But the following error came out when I tested with my own parameters

File "<ipython-input-30-ea35b3db9fab>", line 3, in <module> standardmodelparams = standardparam, hyperparams = hyper_param)

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/toolkits/modelparametersearch.py", line 285, in modelparametersearch return _gl.deploy.job.create([pipeline], name=name, environment=environment)

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/deploy/job.py", line 69, in create environment = environment.clone()

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/deploy/context.py", line 52, in clone return copy.deepcopy(self)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(HTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0