Number of network bytes sent by the algorithm

User 350 | 8/18/2014, 7:19:54 PM

Hi,

Is there is a convenient way to print out the total number of bytes sent by an algorithm by all the cluster's machines together?

Thanks a lot, Michael.

Comments

User 586 | 8/18/2014, 8:17:59 PM

Looking at the <a href="http://docs.graphlab.org/classgraphlab11distributed__control.html">distributedcontrol</a> object, which manages distributed communication, there is a networkmegabytessent() which includes all bytes including headers and control, and bytessent() which excludes headers and control overhead


User 350 | 8/18/2014, 11:31:46 PM

Great, thanks a lot.


User 350 | 8/22/2014, 10:43:07 PM

It looks like the dc.networkmegabytessent() gives a strange number. For example, I execute an algorithm on 25 machines. The output of dc.networkmegabytessent() gives ~ 830MB, while the "globallogger().setloglevel(LOGINFO)" prints out "Network Sent: 159416618" for each of the 25 machines, i.e., 25*159MB ~= 4GB.

What is correct? How I should treat these statistics?

Thanks a lot, Michael.