Memory error calling item_similarity_recommender.create

User 3237 | 2/21/2016, 10:02:37 PM

Saw another post with this same issue, which seemed to have been resolved, but there was no comment as to how?

This is running on AWS EC2 using a training data set of 9,952 records. It worked the first time (while replicating what I was watching in the courera course) but threw this memory error when I tried it again in a new ipython notebook as part of the assignment.

Any advice would be appreciated.


PROGRESS: Recsys training: model = itemsimilarity PROGRESS: Warning: Ignoring columns songid, listencount, title, artist; PROGRESS: To use one of these as a target column, set target = <columnname> PROGRESS: and use a method that allows the use of a target. PROGRESS: Preparing data set. PROGRESS: Data has 893580 observations with 66085 users and 9952 items. PROGRESS: Data prepared in: 2.07706s


MemoryError Traceback (most recent call last) <ipython-input-14-d0481e3cf617> in <module>() ----> 1 personalizedmodel = graphlab.itemsimilarityrecommender.create(traindata, userid = 'userid', item_id = 'song')

/usr/local/lib/python2.7/dist-packages/graphlab/toolkits/recommender/itemsimilarityrecommender.pyc in create(observationdata, userid, itemid, target, userdata, itemdata, nearestitems, similaritytype, trainingmethod, threshold, onlytopk, randomseed, verbose) 178 'onlytopk': onlytopk} 179 --> 180 response = graphlab.toolkits.main.run('recsystrain', opts, verbose) 181 return ItemSimilarityRecommender(response['model']) 182

/usr/local/lib/python2.7/dist-packages/graphlab/toolkits/main.pyc in run(toolkitname, options, verbose, showprogress) 58 try: 59 starttime = time.time() ---> 60 (success, message, params) = unity.runtoolkit(toolkitname, options) 61 end_time = time.time() 62 except:

graphlab/cython/cyunity.pyx in graphlab.cython.cyunity.UnityGlobalProxy.run_toolkit()

graphlab/cython/cyunity.pyx in graphlab.cython.cyunity.UnityGlobalProxy.run_toolkit()

MemoryError: std::bad_alloc

Comments

User 19 | 2/22/2016, 5:44:18 PM

Hi esmit,

Sorry you hit this error. I agree it is a bit surprising.

How much memory does the EC2 machine have? I would recommend trying this on an m3.2xlarge. If you are trying to use a micro instance, it may not have the amount of memory that the model expects to be available.

Cheers, Chris