User 512 | 10/15/2015, 3:37:54 PM
I was parsing a pretty big text file. At the end Graphlab was stopped because of IOError and ValueError:
PROGRESS: Finished parsing file 20151010.gz PROGRESS: Parsing completed. Parsed 170757196 lines in 2459.32 secs. PROGRESS: Less than 48 successfully started. Using only 35 workers. PROGRESS: All operations will proceed as normal, but lambda operations will not be able to use all available cores. PROGRESS: To help us diagnose this issue, please send the log file to email@example.com. PROGRESS: (The location of the log file is printed at the start of the GraphLab server). [INFO] Stopping the server connection. [WARNING] <type 'exceptions.IOError'> [WARNING] <type 'exceptions.ValueError'>
Below is my Graphlab settings: gl.setruntimeconfig('GRAPHLABDEFAULTNUMPYLAMBDAWORKERS', 48) gl.setruntimeconfig('GRAPHLABCACHEFILELOCATIONS', '/share01:/share02:/share03:/share04:/share05') gl.setruntimeconfig('GRAPHLABFILEIOMAXIMUMCACHECAPACITY', 2147483648 * 35) # 70GB gl.setruntimeconfig('GRAPHLABFILEIOMAXIMUMCACHECAPACITYPER_FILE', 134217728 * 500) # ~50GB
How could I fix this error? Thanks!