Is Dynamic Vertex Migration possible in Graphlab or powergraph?

User 1429 | 5/14/2015, 7:09:55 PM

Sir, We will run the algorithm in supersteps in garphlab (powergarph). But after each iteration all the machines in cluster are getting synchronized. If workload on any system increases it will take more time. Because of this all machines in cluster should wait until heavy workload machine completes its job. Is it possible to migrate the vertices from over utilized machine to less utilized machines during run time. If it possible how can we do it powergraph or graphlab. What is the best way to identify these systems in powergarph or graphlab. Please reply as soon as possible.

Thank You

Comments

User 1429 | 5/14/2015, 7:12:30 PM

@srikris sir please help me


User 91 | 5/14/2015, 7:57:04 PM

In general, it is tricky to do what you are asking.

How large is your graph? Recently, we were able to run Pagerank on the largest publicly known graph i.e the Common Crawl Web graph ( 128 billion edges relating 4 billion nodes) at around 10 mins per iteration on a single machine using GraphLab Create. It is entirely possible that you might not need multiple machines.


User 1429 | 5/14/2015, 8:10:58 PM

@srikris Sir, I am working on dynamic work load balancing in distributed graph analytic frameworks. I have read this vertex migration concept in Mizan framework. Is this concept can be implemented in powergraph ?? how workload balancing is being done in powergraph ? only intelligent or optimal graph partitioning(greedy approach)??. How workload can be monitored during run time?


User 1429 | 5/15/2015, 5:54:06 PM

@srikris Sir please help me. Is Dynamic graph partitioning is possible in powergraph or not? If possible can you please explain me how it can be done. Please reply as soon as possible. Thank You


User 15 | 5/18/2015, 6:28:40 PM

Hi @harsha,

While it is an interesting problem, the GraphLab research group did not take this problem on, so it is not supported in PowerGraph. I'm sure you could implement it in PowerGraph, but we are not currently adding new features to that codebase, so it would be up to you to implement it. However, reading over your question again, you may be interested in PowerGraph's asynchronous execution mode.

Currently GraphLab Create does all graph computations on a single node, so it is also unsupported in GraphLab Create as this only makes sense in a distributed environment.

Evan


User 1429 | 5/20/2015, 5:27:33 PM

@EvanSamanas Thank you sir