RuntimeError: Runtime Exception. Cannot evaluate lambda. Lambda workers did not start

User 3500 | 3/12/2016, 12:19:22 PM

Hi,

I tried various python versions..,I upgraded python from 2.7.8 to 2.7.11. I tried graphlab-create 1.8.3 and also graphlab-1.8.4

Even then I am facing the same problem..pylambda worker not starting..

.

Could you please look into this issue...

Comments

User 1207 | 3/14/2016, 9:24:39 PM

Hello @yaseen,

Sorry you are hitting this issue. We're looking in to it, but I don't yet know what's going on. The windows environment is very tricky to work with robustly, and it's likely a corner case in your configuration we didn't account for. We'll get to the bottom of it.

Can you tell me how your environment is set up? Are you using conda or other libraries? Did you install GLC using PIP or with the Dato launcher?

If with PIP, could you tell me if version 1.8.2 works? There were some changes between 1.8.2 and 1.8.3 that may have affected this (although they shouldn't have). If 1.8.2 works for you, this will help me debug it on my end.

Any other information you can provide would be helpful. (Also, if you find it works through installing it with the dato launcher at https://dato.com/download/install-graphlab-create.html, then that would be very helpful information).

Thanks! -- Hoyt


User 3134 | 3/17/2016, 12:34:09 PM

Hi,

Observed the same issue in Databricks environment. Using graphlab-create==1.8.4 with Ubuntu 15.10 instances.

Code: data = graphlab.SFrame(range(1000)).apply(lambda a: a['X1'] % 2)

Full trace:

! RuntimeError Traceback (most recent call last) ! <ipython-input-10-d22f2dacf946> in <module>() ! ----> 1 data = graphlab.SFrame(range(1000)).apply(lambda a: a['X1'] % 2) !
! /databricks/python/local/lib/python2.7/site-packages/graphlab/datastructures/sframe.pyc in apply(self, fn, dtype, seed) ! 2607 ! 2608 with cythoncontext(): ! -> 2609 return SArray(proxy=self.proxy.transform(fn, dtype, seed)) ! 2610 ! 2611 def flatmap(self, columnnames, fn, columntypes='auto', seed=None): !
! /databricks/python/local/lib/python2.7/site-packages/graphlab/cython/context.pyc in exit(self, exctype, excvalue, traceback) ! 47 if not self.showcythontrace: ! 48 # To hide cython trace, we re-raise from here ! ---> 49 raise exctype(excvalue) ! 50 else: ! 51 # To show the full trace, we do nothing and let exception propagate ! RuntimeError: Runtime Exception. Unable to evaluate lambdas. Lambda workers did not start.

Attaching the log for reference.


User 3134 | 3/17/2016, 12:50:33 PM

Not sure whether this will be of help.

When I restart the cluster and run, the code runs fine for the first time. However, some warnings will be output.

! Less than 8 successfully started. Using only 1 workers. ! All operations will proceed as normal, but lambda operations will not be able to use all available cores. ! To help us diagnose this issue, please send the log file to product-feedback@dato.com. ! (The location of the log file is printed at the start of the GraphLab server).

If I run the same code again - BAM!! - Hit the error.

Tried with v1.8.2 as well. Same issue.


User 1190 | 3/17/2016, 5:22:19 PM

How much memory does your machine have?