Invalid argument (stream_engine.cpp:790,836)

User 2594 | 12/7/2015, 3:46:14 AM

I get an error: [INFO] GraphLab Server Version: 1.7.1 Invalid argument (stream_engine.cpp:790) Invalid argument (stream_engine.cpp:836) Abort trap: 6 when I run this code: ` inOrder = features.data.values() # a dictionary of key/value pairs, where the values are objects I created

fs = graphlab.SFrame({ "feature": map( lambda x: x.name, inOrder ), "instances": map( lambda x: x.instances, inOrder ), "samples": map( lambda x: x.samples, inOrder ), "positive": map( lambda x: x.positive, inOrder ), "negative": map( lambda x: x.negative, inOrder ), "neutral": map( lambda x: x.neutral, inOrder ) })

fs.save( 'feature-set', format='binary' ) ` Any ideas?

Comments

User 2594 | 12/7/2015, 3:53:33 AM

It may not be in the saving, it may be in the loading of the source data, which I'm iterating over. Are there any upper limits to data size in an individual SFrame element? The data I'm trying to read includes some big dictionaries...


User 15 | 12/8/2015, 1:00:53 AM

Individual SFrame elements are limited by your memory, and potentially less than that depending on code that makes assumptions about how many elements it can load in to memory. Gigantic elements really aren't a use case we test as it's quite uncommon.

As a rule, a lot of these errors can appear like they're happening during save, but the SFrame is lazy, so it may be executing any number of things you did before the save, and the save is just forcing it to act on all of those operations.

I'm confused by the stream_engine.cpp message though...we don't have a source file in our repository called that. Are you sure that message is coming from GraphLab Create?

Evan


User 2594 | 12/8/2015, 9:59:03 AM

I think I just ran out of memory. I converted all sequences of digits to that many (#) characters, which greatly reduced the number of unique combinations that could be produced, and the problem went away. Not a bug! :)