Error when saving SFrame - Cannot convert ... to float

User 512 | 10/26/2015, 10:15:33 PM

I have a SFrame with all columns defined as str format, when I save this SFrame, Graphlab shows a bunch of error messages: 1445895284 : ERROR: (convert_value_to_output_type:180): Cannot convert 127.0.0.1 to float 1445895284 : ERROR: (convert_value_to_output_type:180): Cannot convert - to float Why would Graphlab try converting str to float? And how can I fix this?

Comments

User 4 | 10/26/2015, 11:13:17 PM

Hi @"Shuning Wu",

This happens commonly due to the type inference used by SFrame and SArray apply functions. The apply function returns new column of data (a new SArray), and by default the type of this new column is inferred from the return value of the lambda or function provided. However, there is a corner case: if the first 100 rows of this return value are all None, then the type cannot be correctly inferred! In this case the type is assumed to be float, but it turns out it could be any type -- the symptom is then what you see printed out later on. The reason you might be seeing this behavior during save is that SFrame and SArray use lazy evaluation to postpone unneeded computation until its result is needed. Upon save, everything needs to be "materialized" (all the postponed computation needs to be performed) so the correct values can be written to disk.

To work around this, you can supply a type hint to the apply functions. Use the keyword argument dtype, as described here: https://dato.com/products/create/docs/generated/graphlab.SArray.apply.html


User 512 | 10/27/2015, 3:53:44 PM

Thanks for the info!