Limiting the number of used cores.

User 2032 | 6/15/2015, 2:46:30 PM

I have asked this question before in a discussion but I think it deserves a separate thread.

Is is possible to limit the number of cpu cores used by graphlab during computation?

Use cases:

  1. Running on a server that also has other things to do. (yep we could use KVM to separate graphlab but it is better to share).
  2. Running on your laptop and you don't want GL to hog all resources and render other apps unresponsive.

I assume it should be somwhere under graphlab.set_runtime_config() but where is this documented? I couldn't find it using search on readthedocs and looking through the source also did not yield any results regarding CPU.

Comments

User 1207 | 6/15/2015, 8:49:56 PM

The number of threads GLC uses internally can be controlled by setting the environment variable OMPNUMTHREADS. It can't be changed once GLC is started, but it can be set at start.

If your shell is bash, setting export OMP_NUM_THREADS=8 will mean each GLC server only uses 8 cores.