Powergraph workload split between cores

User 350 | 2/15/2015, 5:40:56 AM


How does powergraph split the load between the cores? I understand that gather operation may be split across the core, but the following behavior seems strange to me.

I run a program on 2 machines with 36 core each. I set powergraph to use 30 on each. First 30 minutes of the execution, I see 3000% utilization on each machine - i.e., 30 cores are utilized. - Then utilization on both machines falls down to 100% - i.e., only one core is utilized.

So, it looks like the program has some computation bottleneck at some node of the graph.

Moreover, when I run the same program on one machine and specify 30 cores, I see the same pattern: initially all 30 cores work and then just one.

What part of a vertex program can cause such a behavior? One utilized core when 1 machine is used, and 2 cores when 2 machines are used.

Thanks! Michael.


User 6 | 2/15/2015, 4:09:25 PM

Power graph locks between the gather apply and scatter stages. I suggest to add some printouts to identify which part of your code runs using a single core. It of course may be related to the graph structure as well.

User 350 | 2/15/2015, 5:47:46 PM

I see... OK, I will add printouts in all the stages (gather, apply, scatter) and will condition on current elapsed time (since I know that the single core execution begins after about 30 mins...). Thanks Danny!