model_parameter_search for gl.recommend.create

User 597 | 9/25/2014, 4:33:09 PM

So I got to know how to use hyper parameter search using graph lab module. But the following code gave me an error:

import graphlab as gl

columns = yahoo_train_SFrame.column_names()
user_id = columns[0]
product_id = columns[1]
values = columns[2]

env = gl.deploy.environment.Local('local_env')
standard_param = {'observation_data': yahoo_train_SFrame, 'user_column': user_id, 'item_column': product_id, 'target_column': values}
hyper_param = {'n_factors':[10,11,12,13,14], 'regularization':[0.000001, 0.00001, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000],
        'nmf':[True, False], 'unobserved_rating_regularization': [0.000001, 0.00001, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000],
        }

gl.toolkits.model_parameter_search(environment = env, model_factory = gl.recommender.create, 
                                   train_set = 'yahoo_train.csv', save_path = 'hyperparameter_search_result.csv', test_set= 'yahoo_test.csv', 
                                   standard_model_params = standard_param, hyper_params = hyper_param)

The Error:

Traceback (most recent call last):

File "<ipython-input-17-e1b1aa1a7606>", line 8, in <module> standardmodelparams = standardparam, hyperparams = hyper_param)

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/toolkits/modelparametersearch.py", line 285, in modelparametersearch return _gl.deploy.job.create([pipeline], name=name, environment=environment)

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/deploy/job.py", line 69, in create environment = environment.clone()

File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/graphlab/deploy/context.py", line 52, in clone return copy.deepcopy(self)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 257, in deepcopydict y[deepcopy(key, memo)] = deepcopy(value, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo)

File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/coHtml�I�M! ��7# ++����FYI: If you are using Anaconda and having problems with NumPyHello everyone,

I ran into an issue a few days ago and found out something that may be affecting many GraphLab users who use it with Anaconda on Windows. NumPy was unable to load, and consequently everything that requires it (Matplotlib etc).

It turns out that the current NumPy build (1.10.4) for Windows is problematic (more info here).

Possible workarounds are downgrading to build 1.10.1 or forcing an upgrade to 1.11.0 if your dependencies allow. Downgrading was easy for me using conda install numpy=1.10.1

Thanks for your attention!

RafaelMarkdown558,824,8414L���4L���179.110.206.156179.110.206.1564P�}��Xj�8\j�1str�"��\j�Xj��\j�8bj�րi�1(׀i��g��b�j����Xj�\j�Xj�8\j�1.hpp(decrementdistributedcounter:787): Distributed Aggregation of likelihood. 0 remaining. INFO: distributedaggregator.hpp(decrementdistributedcounter:793): Aggregate completion of likelihood Likelihood: -3.22336e+08 INFO: distributedaggregator.3HLABDISABLELAMBDA_SHM"] = "1" os.environ["GRAPHLABFORCEIPCTOTCP_FALLBACK"] = "1" import graphlab as gl

3. Test out your lambda worker code in this environment. If it works, then you can make the above configuration permanent by running:

gl.sys_util.write_config_file_value("GRAPHLAB_DISABLE_LAMBDA_SHM", "1")
gl.sys_util.write_config_file_value("GRAPHLAB_FORCE_IPC_TO_TCP_FALLBACK", "1")

Note that this can be undone by setting these to "0" instead of "1", or by editing the file given by gl.sys_util.get_config_file().

4. If the lambda workers do not work after trying step 1, then there are two things we would very much appreciat

Comments

User 16 | 9/26/2014, 5:35:02 PM

The values for the trainset and testset parameters must be absolute paths. They must also point to SFrames not to CSV files. Likewise save_path must be an absolute path and the result will be saved as an SFrame not a CSV.

Please let me know if you have any other problems or questions.

Toby