User 4898 | 4/16/2016, 4:37:41 PM
Hello, I am having a difficult time loading a sparse matrix. I have written the code below:
mat = sparse_mat.toarray()
df = pd.DataFrame(mat)
sf = gl.SFrame(df)
the last step is the bottle neck. The previous steps took ~1min to perform and the last step is taking over an hour and still running. I am running this on a m4.10xlarge aws instance that has 160 GiB of ram. The data frame has the following dimensions: rows = 12821, cols = 1517490. Any ideas?