TypeError: can't pickle thread.lock objects (when running graphlab.deploy.job.create())

User 5242 | 5/31/2016, 2:37:40 PM

I was creating two jobs: one job that calculates recommendations from csv data, and another job that persist recommendations to a local sqlite database. While the first job works every time, the second one never works and gives me an error: TypeError: can't pickle thread.lock objects

How do I fix this? Thanks!


User 2014 | 5/31/2016, 10:49:27 PM

Hi Junhao,

We would need more information about what you are doing.

  • Are you jobs running in local mode or a remote cluster?
  • If in cluster, what do you mean by local sqlite? If local, then OK :smile:
  • Are you using the required packages annotation?

sqlite is not picklable by itself, but you can make it a dependency. Maybe this would help: https://dato.com/learn/userguide/deployment/pipeline-dependencies.html

Also: https://dato.com/products/create/docs/generated/graphlab.deploy.required_packages.html

Thanks, Antoine