User 4000 | 3/25/2016, 8:45:44 AM
I am running the k-means routine from the python bindings on a file with 66 Million samples and 8 dimensions. I have a 4 NUMA-node, 48 core, 1TB RAM linux box I'm running on. I find that a very large portion of time spent is system time and not computation. I checked the runtime configuration here and for every item that affects performance I have very large values (several hundred GBs for example when compared to the file size on disk of 6.2 GB. It is taking 20+ min to compute 10 clusters when I would expect ~5 min. Is there a list of other configurations I should check to get the best better performance?