How to test node failure in Powergraph

User 1768 | 9/22/2015, 9:29:43 AM

Hi all, I need your help. I would like to know how I can test my algorithm for node failure in Powergraph? For example I want to test, if 30 % of vertices fail to reply the massages, what will be the result of algorithm. I found in one of the articles that there is an Activate(v) and Activate.all() in power graph. But, I count find any example how to use it? I will be appreciated if you guide me,

Comments

User 1592 | 9/22/2015, 5:04:31 PM

Hi PowerGraph is not designed for fault tolerant execution, and thus if a node fails you will need to restart the computation on all nodes.

The only mechanism which is designed for fault tolerance is the snapshot mechanism (See https://www.usenix.org/system/files/conference/osdi12/osdi12-final-167.pdf - section 7.5)

To operate the snapshot mechanism you need to execute the synchronous engine. Here is the command line option documentation:

snapshot_interval: (default: -1) If set to a positive value, a snapshot
is taken every this number of iterations. If set to 0, a snapshot
is taken before the first iteration. If set to a negative value,
no snapshots are taken. A snapshot is a binary dump of the graph.

snapshot_path: If snapshot_interval is set to a value >=0,
this option must be specified and should contain a target basename 
for the snapshot. The path including folder and file prefix in 
which the snapshots should be saved

User 1768 | 9/26/2015, 8:11:54 AM

Thanks Danny for your attention, as I read somewhere, Initially all vertices are active in PowerGraph. PowerGraph executes the vertex-program on the active vertices until none remain. The order of executing activated vertices is up to the PowerGraph execution engine. Once a vertex-program completes the scatter phase it becomes inactive until it is reactivated. Vertices can activate themselves and neighboring vertices. Then, there need to be a solution to activate some of the vertices, instead of all. I will be appreciated, if you can guide me.


User 1592 | 9/27/2015, 6:32:29 AM

Here is an example of dynamic style traversal on a graph: https://github.com/dato-code/PowerGraph/blob/18c21033d77208cedac7661e4fcf35698abb4021/toolkits/graph_analytics/kcore.cpp All nodes are initially active, but nodes are removed based on their graph properties