Number of network bytes sent by the algorithm

User 350 | 8/18/2014, 7:19:54 PM


Is there is a convenient way to print out the total number of bytes sent by an algorithm by all the cluster's machines together?

Thanks a lot, Michael.


User 586 | 8/18/2014, 8:17:59 PM

Looking at the <a href="">distributedcontrol</a> object, which manages distributed communication, there is a networkmegabytessent() which includes all bytes including headers and control, and bytessent() which excludes headers and control overhead

User 350 | 8/18/2014, 11:31:46 PM

Great, thanks a lot.

User 350 | 8/22/2014, 10:43:07 PM

It looks like the dc.networkmegabytessent() gives a strange number. For example, I execute an algorithm on 25 machines. The output of dc.networkmegabytessent() gives ~ 830MB, while the "globallogger().setloglevel(LOGINFO)" prints out "Network Sent: 159416618" for each of the 25 machines, i.e., 25*159MB ~= 4GB.

What is correct? How I should treat these statistics?

Thanks a lot, Michael.