GraphLab with YARN

User 331 | 6/1/2014, 12:31:06 PM

I am very new in BigData and similar problems and I am wondering if GraphLab is working under YARN or Mesos, and if not does it mean that I can only use GraphLab nodes only for GraphLab - I cannot install there for example spark or MapReduce, or maybe it can just run as a normal different process on many nodes?

In other words if I want to have GraphLab and MapReduce/Spark and use all of them how I should set up them on nodes - It would be great to minimise EC2 costs so lower number of nodes is better (as long as it do what is needed to do).

Thanks in advance for any help:)


User 6 | 6/1/2014, 4:26:46 PM

Hi, As seen here: Hadoop integration is coming soon. We will notify on our website/forum once it is released. Our goal is to allow GraphLab to exist in the Hadoop echo system and to allow the same cluster to run different GraphLab/ map reduce jobs.

User 331 | 6/2/2014, 11:41:50 AM

Danny thanks for answer, but I have few more questions: 1. Does integration with Hadoop will be part of open source Graphlab? 2. You mentioned that this integration is coming soon - may you be more specific - like 1,3,6 months or maybe later (just estimation, I am aware that it may change).

User 6 | 6/3/2014, 12:35:53 PM

1) GraphLab open source already supports read and write from HDFS. 2) Yarn integration is planned for GraphLab Create. Please follow our website for a notice when it is ready