unity server processes not utilizing CPU

User 2356 | 10/12/2015, 7:14:38 AM

The gl unity server classifier process is running in bursts most of the time it uses 0.7% of CPU , and it clearly seems that it gets stuck or remaining idle for no reason. currently training on 1 lakh samples with 100+ feature columns . For larger datasets it just throws up Communication Failure :113 errors and exits. So why it claims that it can run on TBs of data?

the top command prints following and more or less same since past 3 days:

32631 root 20 0 6872096 4.664g 152 S 1.3 14.9 373:58.71 unityserver
460 root 20 0 290968 13552 1796 S 0.7 0.0 1:32.17 ipython
495 root 39 19 0 0 0 S 0.3 0.0 835:11.91 kipmi0
29463 root 20 0 12.431g 217564 0 S 0.3 0.7 59:19.26 unity
server

Our config: 32 GB RAM , 8 Core haswell , 400GB free space in 3TB HDD

Comments

User 1190 | 10/12/2015, 12:01:28 PM

Hi @abby,

Sorry to hear that you encountered issues with GraphLab Create. We have tested our classifiers on datasets with similar size: 100K samples, and 100+ features.

Would you mind providing a code sample for us to take a look?

Please also provide the following information about your dataset. What classifier are you using? Are features numerical or categorical? (How many numerical? how many categorical?) Is there any feature engineering steps?

Thanks, -jay


User 2356 | 10/12/2015, 12:17:45 PM

only categorical features all of them are text, running boosted tree classifeir using the create method. @JayGu


User 2356 | 10/13/2015, 6:34:09 AM

@JayGu Do let me know about this issue as I see even others have also posted about this issue of underutilization of both memory and CPU. The code is a simple create command to which I pass the large dataset. It is not working for 3 lac dataset size and throwing memory errors. Managed to get it working with 1 lakh samples only.


User 2356 | 10/14/2015, 6:01:37 AM

@piotr to check out this thread