How to observe the GraphChi's performance in the right way?

User 429 | 7/8/2014, 9:25:06 AM

I have a question about the system: Do users not supposed to set the number of shards manually by just let "shards"( in the main function) to be a fixed number? I try to set different values to nshards instead of let it computed by the preprocessing function but it crashed during the make period. The reason I want to change the value of shards is I want to observe the system's performance on memory using. And I also did not found any info about how much memory does the one time of running totally use.


User 6 | 7/8/2014, 2:44:05 PM

It depends on which algorithm you use. Some algorithms take membudget_mb as a paramter which defines how much data is pulled to memory. nshards=1 forces all data to reside in memory while nshards=2,3,4... load a chunk of the data each time.