SFrame read_csv treats quotes differently in version 1.3?

User 552 | 2/19/2015, 4:22:44 PM

Is there a change in version 1.3 related to how the read_csv method interprets fields with quotes. Till the last version if I had a line like: "10","20" in a csv file and I provided the right column hints, then it would interpret the fields as integers 10 and 20. But now the CSV parsing fails. I need to make the line 10,20 (without quotes) for the file to be parsed. Is there a workaround available?

Comments

User 1189 | 2/26/2015, 12:09:05 AM

Hi,

There were some deep-ish changes to the csv parser to improve type inference behavior, and while I have quite a large collection of tests, your case appeared to have leaked through. I think I never intended your case to have worked at all :-)

For now, you should be able to parse without column hints and it should interpret the columns as strings, and then you can do a type conversion with sf['colname'].astype(int).

Performing deeper interpretation on the contents of quoted fields (rather than just letting them be strings) is definitely a TODO on my list.

Yucheng