User 11 | 3/3/2014, 5:05:25 PM
I was running SSSP algorithm for multiple source vertices. I basically load the graph once and then iterate on all given source vertices. After running the engine for each source vertex I have a command to initialize all vertices distance by "graph.transformvertices(initvertex);".
I noticed that graphlab initially allocates 11.5 GB of memory (in the whole cluster) but eventually after processing several source vertices, the memory usage decreased until it reached about 6 GB after processing about 150 source vertices. This is an amazing performance. My guess is probably Graphlab learn what is the workload and probably re-distribute edges on the cluster nodes to reduce memory (replication). My question is, if this is the case how does GraphLab do this? Is there an internal engine that monitor workload?
If my guess is not accurate, do you have an explanation for this behavior?