Runtime Exception. vector::_M_default_append while "array.apply"

User 2159 | 8/10/2015, 1:22:08 PM

Hi, I am trying to run this simple code, that should add a new column to my SFrame the contains the first 3 letters from the "last name" field.

frame['init_lastName'] = frame['lastName'].apply(lambda x: x[:3])

I got this error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-8-b4e636eb89f2> in <module>()
----> 2 frame['init_lastName'] = frame['lastName'].apply(lambda x: x[:3])

/usr/local/lib/python2.7/dist-packages/graphlab/data_structures/sframe.pyc in __setitem__(self, key, value)
   3358             if not key in self.column_names():
   3359                 with cython_context():
-> 3360                     self.add_column(sa_value, key)
   3361             else:
   3362                 # special case if replacing the only column.

/usr/local/lib/python2.7/dist-packages/graphlab/cython/context.pyc in __exit__(self, exc_type, exc_value, traceback)
     47             if not self.show_cython_trace:
     48                 # To hide cython trace, we re-raise from here
---> 49                 raise exc_type(exc_value)
     50             else:
     51                 # To show the full trace, we do nothing and let exception propagate

RuntimeError: Runtime Exception. vector::_M_default_append

What am I doing wrong? Thanks! D

Comments

User 1359 | 8/10/2015, 9:14:45 PM

Hi Davud,

I am looking into this. Could you give me a few more details?

What version of GraphLab Create are you running? gl.version
How large is the sframe?
Can you post a small code example that reproduces the error?

Thanks Dick

User 2159 | 8/11/2015, 7:10:56 AM

Thanks!

1) The current version i am using is '0.325' (internal Dato version, due to error we had with MySQL driver) 2) The sframe is big... ~4,200,000 records. (i found the number while querying my db, len(frame) is running for more than a hour and and than throw the same error!) 3) On tiny part of the sframe (~10,000 records) it worked with no problem.

another small code that reproduced the error:

len(frame)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-3a8a6cb3b6ec> in <module>()
----> 1 len(frame)

/usr/local/lib/python2.7/dist-packages/graphlab/data_structures/sframe.pyc in __len__(self)
   2295         Returns the number of rows of the sframe.
   2296         """
-> 2297         return self.num_rows()
   2298 
   2299     def __copy__(self):

/usr/local/lib/python2.7/dist-packages/graphlab/data_structures/sframe.pyc in num_rows(self)
   2355         num_columns
   2356         """
-> 2357         return self.__proxy__.num_rows()
   2358 
   2359     def num_cols(self):

graphlab/cython/cy_sframe.pyx in graphlab.cython.cy_sframe.UnitySFrameProxy.num_rows()

graphlab/cython/cy_sframe.pyx in graphlab.cython.cy_sframe.UnitySFrameProxy.num_rows()

RuntimeError: Runtime Exception. vector::_M_default_append

User 1189 | 8/11/2015, 5:06:56 PM

I am guessing we are hitting memory issues. is the machine you are running on relatively low on memory? What is the platform you are running on (Mac/Linux) and what is the system configuration? We increased the default caching limits this version and it is somewhat more aggressive with memory utilization.

There is a faster non-lambda version if you want to slice the values. Look up SArray.subslice frame['init_lastName'] = frame['lastName'].subslice(start=0, stop=3)

User 2159 | 8/12/2015, 2:39:30 PM

Hi Yucheng, The machine i am running on has 16 GB memory. OS: Ubuntu 12.04.5 LTS. Any more information can help?

The SArray.subslice returns the same error...

User 1189 | 8/12/2015, 4:45:09 PM

Hi,

I am unable to reproduce the issue.

On startup you may see a line of this sort: [INFO] Start server at: ipc:///... - Server binary: .... - Server log: /tmp/graphlab_server_1439397958.log

Can you attach the file [Server log].0 (for instance /tmp/graphlabserver1439397958.log.0 above)

User 1189 | 8/12/2015, 4:47:33 PM

Hi,

How many columns is your SFrame?

User 2159 | 8/13/2015, 12:06:19 PM

len(frame.column_names())

returns: 14

User 1189 | 8/17/2015, 6:20:28 PM

Sorry, after much investigation we are unable to reproduce this error. Can you attach the log file?

On startup you may see a line of this sort: [INFO] Start server at: ipc:///... - Server binary: .... - Server log: /tmp/graphlab_server_1439397958.log Can you attach the file [Server log].0 (for instance /tmp/graphlabserver1439397958.log.0 above)

User 2159 | 8/27/2015, 2:46:39 PM

Sorry for another late response. Is it possible to send you the log file on a private mail?

Meanwhile, here are the errors from the log file:

1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull 1439470867 : ERROR: (operator():56): Check failed (/var/lib/jenkins/jobs/Dato-Dev-Continuous-Build-Linux/workspace/osssrc/sframequeryengine/operators/generalizedunionproject.hpp:56): allnull

User 1189 | 8/31/2015, 6:19:55 PM

Sure, please email to ylow@dato.com.

Thanks! Yucheng