How can I determine the hardware that is needed for build a graph model using PowerGraph?

User 1043 | 12/11/2014, 1:11:30 AM

I plan to do some graph model with loopy belief propagation in PowerGraph. I am designing the experiment to determine how many nodes and edges I want to include for training purpose. I don't know how I can estimate the size of the hardware and memory given a sample size. Any suggestions?

Comments

User 6 | 12/11/2014, 4:11:05 PM

PowerGraph has to store the full problem in memory. The easiest way is to experiment, start with a single computing node and see if the problem fits your memory. we typically measure the problem size by the number of edges. Problems with roughtly 100M edges can fit a single machine - but this depends on your RAM and setup details.

GraphLab Create does not yet implement LBP but there the problem is read from disk, so you can scale to much larger problems using a single computing node.