Seg Fault when importing Graphlab

User 3211 | 2/24/2016, 7:39:13 AM

I'm getting a segmentation fault when importing graphlab. Graphlab Create was installed on this linux machine using python 2.7.10 and GCC 4.8.5. This is the error message I'm getting after using gdb to see the stack trace of the seg fault:

Program received signal SIGSEGV, Segmentation fault. 0x000000000029aea6 in ?? () Missing separate debuginfos, use: debuginfo-install atlas-3.8.4-2.el6.x8664 bzip2-libs-1.0.5-7.el60.x8664 expat-2.0.1-11.el62.x8664 fontconfig-2.8.0-5.el6.x8664 freetype-2.3.11-15.el66.1.x8664 glibc-2.12-1.166.el67.7.x8664 keyutils-libs-1.4-5.el6.x8664 krb5-libs-1.10.3-42.el6.x8664 libX11-1.6.0-6.el6.x8664 libXau-1.0.6-4.el6.x8664 libXft-2.3.1-2.el6.x8664 libXrender-0.9.8-2.1.el6.x8664 libcomerr-1.41.12-22.el6.x8664 libffi-3.0.5-3.2.el6.x8664 libgcc-4.4.7-16.el6.x8664 libgfortran-4.4.7-16.el6.x8664 libselinux-2.0.94-5.8.el6.x8664 libstdc++-4.4.7-16.el6.x8664 libuuid-2.17.2-12.18.el6.x8664 libxcb-1.9.1-3.el6.x8664 openssl-1.0.1e-42.el67.2.x8664 tcl-8.5.7-6.el6.x8664 tk-8.5.7-5.el6.x8664 zlib-1.2.3-29.el6.x8664 (gdb) backtrace

0 0x000000000029aea6 in ?? ()

1 0x00007fffecc6328e in ?? ()

from /ibrix/packages/python/2.7.10/lib/python2.7/site-packages/graphlab/libunity_shared.so

2 0x00007fffeb705e46 in _init ()

from /ibrix/packages/python/2.7.10/lib/python2.7/site-packages/graphlab/libunity_shared.so

3 0x0000000000000009 in ?? ()

4 0x000000334dc0e625 in dlinit_internal () from /lib64/ld-linux-x86-64.so.2

5 0x000000334dc12e95 in dlopenworker () from /lib64/ld-linux-x86-64.so.2

6 0x000000334dc0e286 in dlcatch_error () from /lib64/ld-linux-x86-64.so.2

7 0x000000334dc1263a in dlopen () from /lib64/ld-linux-x86-64.so.2

8 0x000000334e800f66 in dlopen_doit () from /lib64/libdl.so.2

9 0x000000334dc0e286 in dlcatch_error () from /lib64/ld-linux-x86-64.so.2

10 0x000000334e80129c in dlerrorrun () from /lib64/libdl.so.2

11 0x000000334e800ee1 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2

12 0x00007ffff7d3bede in PyImportGetDynLoadFunc ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

13 0x00007ffff7d1920e in PyImportLoadDynamicModule ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

14 0x00007ffff7d16de9 in import_submodule ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

15 0x00007ffff7d17ae2 in PyImport_ImportModuleLevel ()

---Type <return> to continue, or q <return> to quit--- from /packages/python/2.7.10/lib/libpython2.7.so.1.0

16 0x00007ffff7cf5fdf in builtin_import ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

17 0x00007ffff7c471c3 in PyObject_Call ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

18 0x00007ffff7cf7a37 in PyEval_CallObjectWithKeywords ()

at Python/ceval.c:3904

19 0x00007ffff7cfa9c6 in PyEval_EvalFrameEx () at Python/ceval.c:2348

20 0x00007ffff7cfe0ed in PyEval_EvalCodeEx () at Python/ceval.c:3267

21 0x00007ffff7cfe222 in PyEval_EvalCode () at Python/ceval.c:669

22 0x00007ffff7d15b8c in PyImport_ExecCodeModuleEx ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

23 0x00007ffff7d15eee in loadsourcemodule ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

24 0x00007ffff7d16de9 in import_submodule ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

25 0x00007ffff7d1706f in ensure_fromlist ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

26 0x00007ffff7d17ca6 in PyImport_ImportModuleLevel ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

27 0x00007ffff7cf5fdf in builtin_import ()

from /packages/python/2.7.10/lib/libpython2.7.so.1.0

28 0x00007ffff7c471c3 in PyObject_Call ()

---Type <return> to continue, or q <return> to quit--- from /packages/python/2.7.10/lib/libpython2.7.so.1.0

29 0x00007ffff7cf7a37 in PyEval_CallObjectWithKeywords ()

at Python/ceval.c:3904

30 0x00007ffff7cfa9c6 in PyEval_EvalFrameEx () at Python/ceval.c:2348

31 0x00007ffff7cfe0ed in PyEval_EvalCodeEx () at Python/ceval.c:3267

32 0x00007ffff7cfe222 in PyEval_EvalCode (Markdown`�I�M! ��7# ++����FYI: If you are using Anaconda and having problems with NumPyHello everyone,

I ran into an issue a few days ago and found out something that may be affecting many GraphLab users who use it with Anaconda on Windows. NumPy was unable to load, and consequently everything that requires it (Matplotlib etc).

It turns out that the current NumPy build (1.10.4) for Windows is problematic (more info here).

Possible workarounds are downgrading to build 1.10.1 or forcing an upgrade to 1.11.0 if your dependencies allow. Downgrading was easy for me using conda install numpy=1.10.1

Thanks for your attention!

RafaelMarkdown558,824,8414L���4L���179.110.206.156179.110.206.1564P�}��Xj�8\j�1str�"��\j�Xj��\j�8bj�րi�1(׀i��g��b�j����Xj�\j�Xj�8\j�1.hpp(decrementdistributedcounter:787): Distributed Aggregation of likelihood. 0 remaining. INFO: distributedaggregator.hpp(decrementdistributedcounter:793): Aggregate completion of likelihood Likelihood: -3.22336e+08 INFO: distributedaggregator.3HLABDISABLELAMBDA_SHM"] = "1" os.environ["GRAPHLABFORCEIPCTOTCP_FALLBACK"] = "1" import graphlab as gl

3. Test out your lambda worker code in this environment. If it works, then you can make the above configuration permanent by running:

gl.sys_util.write_config_file_value("GRAPHLAB_DISABLE_LAMBDA_SHM", "1")
gl.sys_util.write_config_file_value("GRAPHLAB_FORCE_IPC_TO_TCP_FALLBACK", "1")

Note that this can be undone by setting these to "0" instead of "1", or by editing the file given by gl.sys_util.get_config_file().

4. If the lambda workers do not work after trying step 1, then there are two things we would very much appreciate you do to help us track down the issue.

4.1. First, execute the following code in a clean python shell, where you have not yet imported graphlab create. At the end of this code, it prints out the path to a zip file that, if you could send it to us, will help us diagnose the issue. Please create a support tick^j�8bj�2�"��bj�^j��bj�8�j�րi�2(׀i����^j�8bj�2Html�pu��128.237.194.163���MN 

Comments

User 19 | 2/24/2016, 4:18:37 PM

Hi jjbrophy,

Can you tell us a bit more about your system? What version of GraphLab Create do you have installed? This will help us investigate the issue.

Thanks, Chris


User 3211 | 2/24/2016, 10:27:57 PM

Certainly. We are running RHEL 6.7, and have used Python 2.7.10, configured with shared libs and 4-byte unicode and built using both GCC 4.4.7 and then with GCC 4.8.5 compilers. The version of GraphLab-Create is 1.8.2, which was installed using pip.


User 19 | 2/24/2016, 11:15:26 PM

Another quick question: Is there anything else that is imported prior to GraphLab Create? Or does this segfault occur in a fresh Python process immediately after import graphlab?

Thanks, Chris


User 3211 | 2/25/2016, 4:55:51 AM

No, not that I know of, I start a python REPL and try importing graphlab, and then get the seg fault.


User 19 | 2/25/2016, 5:55:35 PM

We've been looking into a few issues around shared libs. We will report back once we know more or we have a fix.

Thanks, Chris


User 3211 | 2/25/2016, 8:10:18 PM

Sounds great, thanks.