error re-starting a previously created ec2 cluster

User 2785 | 12/23/2015, 10:04:33 PM

I created a dato distributed ec2 cluster and am having trouble re-connecting to it with this code:

` import graphlab as gl gl.aws.set_credentials('this', 'that')

print "loading env" mycluster=gl.deploy.ec2cluster.load(s3path='s3://com.urbanairship.coredata-emr/lisagraph_lab')

print "printing mycluster: ", mycluster

print "starting env" my_cluster.start() `

I get this error:

` $ python gbm0501201506012015ec2.py time started loading env [INFO] 1450907888 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450907888 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450907888 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-55331 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1450907888.log [INFO] GraphLab Server Version: 1.7.1 printing mycluster: S3 State Path: s3://com.urbanairship.coredata-emr/lisagraphlab EC2 Config : [instancetype: m3.xlarge, region: us-east-1, awsaccesskey: xxx] Num Hosts : 1 Stopped starting env [INFO] 1450907904 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450907904 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450907904 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

Traceback (most recent call last): File "gbm0501201506012015ec2.py", line 40, in <module> mycluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 233, in start self.idleshutdowntimeout File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 355, in startcommanderhost producttype = ProductType.DatoDistributed) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 420, in ec2factory subnetid = subnetid, securitygroupid = securitygroupid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 365, in setupsecuritygroup securitygroupname, securitygroupid, subnetid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 333, in getsecuritygroupconfig raise Exception('Error: No Subnet inside VPC ' + str(securitygroup.vpc_id)) Exception: Error: No Subnet inside VPC None [INFO] Stopping the server connection.

`

I had initially set up this ec2 cluster via this code:

` import graphlab as gl

gl.aws.set_credentials(this', 'that')

myconfig = gl.deploy.Ec2Config( awsaccesskeyid='this', awssecretaccesskey='that', region='us-east-1', securitygroup='lisasdefault6', subnet_id='subnet-eea97683' )

mycluster = gl.deploy.ec2cluMarkdown`�I�M! ��7# ++����FYI: If you are using Anaconda and having problems with NumPyHello everyone,

I ran into an issue a few days ago and found out something that may be affecting many GraphLab users who use it with Anaconda on Windows. NumPy was unable to load, and consequently everything that requires it (Matplotlib etc).

It turns out that the current NumPy build (1.10.4) for Windows is problematic (more info here).

Possible workarounds are downgrading to build 1.10.1 or forcing an upgrade to 1.11.0 if your dependencies allow. Downgrading was easy for me using conda install numpy=1.10.1

Thanks for your attention!

RafaelMarkdown558,824,8414L���4L���179.110.206.156179.110.206.1564P�}��Xj�8\j�1str�"��\j�Xj��\j�8bj�րi�1(׀i��g��b�j����Xj�\j�Xj�8\j�1.hpp(decrementdistributedcounter:787): Distributed Aggregation of likelihood. 0 remaining. INFO: distributed_ag

Comments

User 2156 | 12/23/2015, 11:23:48 PM

Hi,

Could you check if you have an associated VPC ID within your securitygroup 'lisasdefault6'? Also, in your subnets config panel, could you check the VPC ID is the same as the one your saw in the securitygroup config panel?


User 2785 | 12/24/2015, 12:24:27 AM

Hi Kaiyu -

There is no VPC ID for the security group "lisadefault6".


User 2156 | 12/24/2015, 12:27:17 AM

I believe that is the reason why it throws an error. Could you try to launch an instance without specifying security_group?


User 2785 | 12/24/2015, 1:21:09 AM

so I tried spinning up a cluster with these settings (note i removed 'security_group' from Ec2Config arguments):

` import graphlab as gl

gl.aws.set_credentials('this', 'that')

myconfig = gl.deploy.Ec2Config( awsaccesskeyid='this', awssecretaccesskey='that', region='us-east-1', subnetid='subnet-eea97683' )

mycluster = gl.deploy.ec2cluster.create( name='lisagraphlab3', s3path='s3://com.urbanairship.coredata-emr/lisagraphlab3', ec2config=my_config ) `

and I get back this error:

` $ python gbm0501201506012015ec2.py [INFO] 1450919306 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450919306 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450919306 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-57937 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1450919306.log [INFO] GraphLab Server Version: 1.7.1 [INFO] 1450919314 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450919314 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450919314 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

Traceback (most recent call last): File "gbm0501201506012015ec2.py", line 25, in <module> ec2config=myconfig File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 83, in create cluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 233, in start self.idleshutdowntimeout File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 355, in startcommanderhost producttype = ProductType.DatoDistributed) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 420, in ec2factory subnetid = subnetid, securitygroupid = securitygroupid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 365, in setupsecuritygroup securitygroupname, securitygroupid, subnetid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 281, in getsecuritygroupconfig subnet_id = subnets[0].id IndexError: list index out of range [INFO] Stopping the server connection. `

so back to square one with not being able to get a ec2 cluster running (been having trouble with this as noted here: http://forum.dato.com/discussion/1542/errors-trying-to-create-an-ec2-instance-via-ec2config-and-ec2-cluster#latest ). Looks like the only way I can create a new cluster is if I include security_group and if I give it a brand new name. But theHTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016


User 2156 | 12/24/2015, 1:29:43 AM

Could you also attach the error message when you load the instance after you successfully create it using a new security_group? It could be a bug and I will sync with the team with your error log to get a fix as soon as possible


User 2785 | 12/24/2015, 2:33:03 AM

ok so here is a brand new ec2 instance 'lisagraphlab_3' with a new security group "lisasdefault8" i created with this code:

` import graphlab as gl

gl.aws.set_credentials('this', 'that')

myconfig = gl.deploy.Ec2Config( awsaccesskeyid='this', awssecretaccesskey='that', region='us-east-1', securitygroup='lisasdefault8', subnet_id='subnet-eea97683' )

mycluster = gl.deploy.ec2cluster.create( name='lisagraphlab3', s3path='s3://com.urbanairship.coredata-emr/lisagraphlab3', ec2config=my_config )

print "setting env" gl.setdistributedexecutionenvironment(mycluster)

my_cluster.stop()

`

the ec2 instance started up successfully but I got an error from the "gl.setdistributedexecutionenvironment(mycluster)" line (error truncated cause it's too large for this forum entry):

` Traceback (most recent call last): File "gbm0501201506012015ec2.py", line 31, in <module> gl.setdistributedexecutionenvironment(mycluster) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/distributedexecutionenvironment.py", line 88, in setdistributedexecutionenvironment ee = create(environment, timeoutinseconds) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/dmlcluster.py", line 40, in create dmlcluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/dmlcluster.py", line 117, in start silent = True) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 267, in submitjob silent = silent) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 465, in submitjob environment.clustercontroller,'submit',parameters=postparms) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/util/fileutil.py", line 830, in fretry return f(*args, **kargs) # last retry File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 304, in makehttppostrequest raise RuntimeError(errormsg) RuntimeError: Server side error: {u'exceptiontype': u"<class 'ssl.CertificateError'>", u'errormessage': u'Could not submit job', u'exception_traceback': [u' File "./clustercontrollerdaemon.py", line 343, in submit\n updatejobstatus(\'Pending\', s3statuspath)\n', u' File "./clustercontrollerdaemon.py", line 213, in updatejobstatus\n s3bucket = s3connection.getbucket(bucketname)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/site-packages/boto/s3/connection.py", line 502, in getbucket\n return self.headbucket(bucketname, headers=headers)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/site-packages/boto/s3/connection.py", line 521, in headbucket\n response = self.makerequest(\'HEAD\', bucketname, headers=headers)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/site-packages/boto/s3/connection.py", line 664, in makerequest\n retryhandler=retryhandler\n', u' File "/home/ubuntu/anaconda/lib/python2.7/site-packages/boto/connection.py", line 1068, in makerequest\n retryhandler=retryhandler)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/site-packages/boto/connection.py", line 942, in mexe\n request.body, request.headers)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/httplib.py", line 1001, in request\n self.sendrequest(method, url, body, headers)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/httplib.py", line 1035, in sendrequest\n self.endheaders(body)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/httplib.py", line 997, in endheaders\n self.sendoutput(messagebody)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/httplib.py", line 850, in sendoutput\n self.send(msg)\n', u' File "/home/ubuntu/anaconda/lib/python2.7/httplib.py", line 812, in send\n self.connect()\n', uHTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0


User 2785 | 12/24/2015, 2:33:07 AM

(this comment is a continuation of the one above it)

step one stop cluster:

my_cluster=gl.deploy.ec2_cluster.load(s3_path='s3://com.urbanairship.coredata-emr/lisa_graph_lab_3') my_cluster.stop()

step two load and start the cluster:

my_cluster=gl.deploy.ec2_cluster.load(s3_path='s3://com.urbanairship.coredata-emr/lisa_graph_lab_3') my_cluster.stop()

trying to restart the cluster gave this error:

` $ python stopcluster.py [INFO] 1450923821 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450923821 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450923821 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-58688 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1450923821.log [INFO] GraphLab Server Version: 1.7.1 print mycluster: S3 State Path: s3://com.urbanairship.coredata-emr/lisagraphlab_3 EC2 Config : [instancetype: m3.xlarge, region: us-east-1, awsaccesskey: AKIAI5SZTCXZE2HDEGVQ] Num Hosts : 1 Stopped [INFO] 1450923828 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450923828 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450923828 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

Traceback (most recent call last): File "stopcluster.py", line 9, in <module> mycluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 233, in start self.idleshutdowntimeout File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 355, in startcommanderhost producttype = ProductType.DatoDistributed) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 420, in ec2factory subnetid = subnetid, securitygroupid = securitygroupid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 365, in setupsecuritygroup securitygroupname, securitygroupid, subnetid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 333, in getsecuritygroupconfig raise Exception('Error: No Subnet inside VPC ' + str(securitygroup.vpc_id)) Exception: Error: No Subnet inside VPC None [INFO] Stopping the server connection.

`


User 2785 | 12/24/2015, 4:16:06 AM

another update:

tried again to create a instance and stop it to see what would appear on the awx ec2 user interface. I'm ultimately hoping to create an instance and use when i need it so loading starting and stopping it. However after creating an instance then running "my_cluster.stop()" the console reports that the instance has been "terminated" (reading from console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:search=i-cf624779;sort=instanceState ):

lisa_graph_lab_1 i-cf624779 m3.xlarge us-east-1a terminated

this was the code i ran:

` import graphlab as gl

gl.aws.setcredentials('this', 'that') gl.setruntimeconfig('GRAPHLABFILEIOALTERNATIVESSLCERTDIR', '1')

myconfig = gl.deploy.Ec2Config( awsaccesskeyid='this', awssecretaccesskey='that', region='us-east-1', securitygroup='lisadefault9', subnet_id='subnet-eea97683' )

mycluster = gl.deploy.ec2cluster.create( name='lisagraphlab1', s3path='s3://com.urbanairship.coredata-emr/lisagraphlab1', ec2config=my_config )

print "mycluster: ", mycluster

gl.setdistributedexecutionenvironment(mycluster)

my_cluster.stop() `

and the output (no errors):

` $ python gbm0501201506012015ec2.py [INFO] 1450929583 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450929583 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450929583 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-59923 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1450929583.log [INFO] GraphLab Server Version: 1.7.1 [INFO] 1450929586 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450929586 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450929586 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Launching an m3.xlarge instance in the us-east-1a availability zone, with id: i-cf624779. You will be responsible for the cost of this instance. [INFO] Waiting for i-cf624779 to start up. [INFO] Deleting keys: [] mycluster: S3 State Path: s3://com.urbanairship.coredata-emr/lisagraphlab1 EC2 Config : [instancetype: m3.xlarge, region: us-east-1, awsaccess_key: AKIAI5SZTCXZE2HDEGVQ] Num Hosts : 1 Status : Running [INFO] Stopping cluster [INFO] Deleting keys: [u'lisa_graph_lab_1/state/pickle_archive', u'lisa_graph_lab_1/state/version'] [INFO] Stopping the server connection. `

i expect that I won't be able to re-load an instance that's been terminated. Any thoughts?


User 1394 | 12/24/2015, 4:36:21 AM

OK this sounds reasonable, but less than ideal.

What is happening here is that the cluster is being stopped (the EC2 instances are terminated but none of the state information about the cluster is lost - that is all captured in S3).

However, you probably want the cluster to be terminated AFTER the job being submitted is completed (or the synchronous execution using graphlab.set_distributed_execution_environment()).

So to enable that type of workflow, here is one suggestion:

For asynchronous execution: `

1. get cluster initialized and ready for work

s3path='s3://com.urbanairship.coredata-emr/lisagraphlab3' cluster = graphlab.deploy.ec2cluster.load(s3path) cluster.start()

2. submit work to the cluster

job = graphlab.deploy.job.create(fntodo_work, environment=cluster)

3. wait for cluster idle timeout to shutdown cluster

` The cluster will automatically shutdown after the final job is completed and the idle timeout (which defaults to 30m when the cluster is created).

For synchronous execution: `

1. get cluster initialized and ready for work

s3path='s3://com.urbanairship.coredata-emr/lisagraphlab3' cluster = graphlab.deploy.ec2cluster.load(s3path) cluster.start() graphlab.setdistributedexecution_environment(cluster)

2. Do work from current session

3. shutdown cluster now that work is done

graphlab.cleardistributedexecution_environment() cluster.stop() ` Does this make sense? The goal should be that the cluster shuts down once the expected work is completed. This is possible with both the async method (using the idle timeout) or the sync method of working with a Dato Distributed cluster.

Let me know if this is unclear of if you have further questions!

Thanks,

Rajat


User 2785 | 12/24/2015, 5:30:56 AM

Hi Rajat,

Thanks for the response! So I set up the 'lisagraphlab1' cluster successfully then stopped it so it theoretically should be ready to be re-loaded. Here is my first attempt with synchronous setup (i.e. setdistributedexecutionenvironment):

my code:

` import graphlab as gl

gl.aws.set_credentials('this', 'that')

print "initialize cluster" cluster=gl.deploy.ec2cluster.load(s3path='s3://com.urbanairship.coredata-emr/lisagraphlab_1')

print "cluster: ", cluster

print "starting cluster" cluster.start()

print "setting env" gl.setdistributedexecution_environment(cluster)

print "reading" sf1 = gl.SFrame.readcsv('s3://com.urbanairship.coredata-emr/toydata/awstoydata_mini.csv')

trainminsends = sf1.groupby('year',{'sendsmin':gl.aggregate.MIN('countsend')}) trainminsends.save('s3://com.urbanairship.coredata-emr/toydata/examplemin_count.csv', format='csv')

cluster.stop() `

error:

` $ python ec2load.py initialize cluster [INFO] 1450933171 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450933171 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450933171 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-60372 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1450933171.log [INFO] GraphLab Server Version: 1.7.1 cluster: S3 State Path: s3://com.urbanairship.coredata-emr/lisagraphlab1 EC2 Config : [instancetype: m3.xlarge, region: us-east-1, awsaccesskey: AKIAI5SZTCXZE2HDEGVQ] Num Hosts : 1 Stopped starting cluster [INFO] 1450933178 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1450933178 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1450933178 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

Traceback (most recent call last): File "ec2load.py", line 11, in <module> cluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 233, in start self.idleshutdowntimeout File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 355, in startcommanderhost producttype = ProductType.DatoDistributed) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 420, in ec2factory subnetid = subnetid, securitygroupid = securitygroupid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 365, in setupsecuritygroup securitygroupname, securitygroupid, subnetid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 333, in getsecuritygroupconfig raise Exception('EHTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0


User 2156 | 12/24/2015, 4:34:09 PM

Hi,

I conformed with the team and the issue you cannot load a cluster after successfully created it (no VPC ID) is indeed a bug, we are pushing a release at the moment. I am sorry for any delay in your work.


User 2156 | 12/24/2015, 5:03:13 PM

You may also consider getting a new ec2 VPC account to unblock your work if it is possible.


User 2785 | 12/26/2015, 8:22:49 PM

Hi- thank you for the confirmation. I'll look into getting an ec2 VPC account on my end. Thanks!


User 2156 | 12/27/2015, 1:37:21 AM

You are welcome! Please keep us posted if there is any other issues


User 2785 | 12/28/2015, 9:59:48 PM

OK confirmed that the following security group ids/names have VPCs. I tried creating a new ec2 instance by either specifying the name or id and came back with these errors:

  1. error after setting the "securitygroupid" in Ec2Config:

` $ python gbm0501201506012015ec2.py [INFO] 1451337816 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1451337816 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1451337816 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

[INFO] Start server at: ipc:///tmp/graphlabserver-10040 - Server binary: /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/unityserver - Server log: /tmp/graphlabserver1451337816.log [INFO] GraphLab Server Version: 1.7.1 [INFO] 1451337819 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1451337819 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1451337819 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assigned to elizabeth.orr@urbanairship.com and will expire on January 16, 2016. Please contact trial@dato.com for licensing options or to request a free non-commercial license for personal or academic use.

Traceback (most recent call last): File "gbm0501201506012015ec2.py", line 27, in <module> ec2config=myconfig File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 83, in create cluster.start() File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/ec2cluster.py", line 233, in start self.idleshutdowntimeout File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/deploy/executionenvironment.py", line 355, in startcommanderhost producttype = ProductType.DatoDistributed) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 420, in ec2factory subnetid = subnetid, securitygroupid = securitygroupid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 365, in setupsecuritygroup securitygroupname, securitygroupid, subnetid) File "/Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/graphlab/connect/aws/ec2.py", line 281, in getsecuritygroupconfig subnet_id = subnets[0].id IndexError: list index out of range [INFO] Stopping the server connection. `

  1. error after setting the "security_group" in Ec2Config:

` $ python gbm0501201506012015ec2.py [INFO] 1451337631 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTFILE to /Applications/anaconda/envs/CHURNSTUFFS/lib/python2.7/site-packages/certifi/cacert.pem 1451337631 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOALTERNATIVESSLCERTDIR to 1451337631 : INFO: (initializeglobalsfromenvironment:282): Setting configuration variable GRAPHLABFILEIOINSECURESSLCERTIFICATECHECKS to 1 This trial license of GraphLab Create is assHTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 21 Jul 2016 23:13:36 GMT Server: Warp/3.2.6 Content-Type: application/json

016A ["37zyefqi2sweveyp","42fn7zeo6v5ui427","66pt5sk2wz2jrbzu","awoljknjigytdyls","cj2lanoogknwopto","cnm3adnh35xmsx3f","ebxs4t2y6xr5izzy","eg5zus2pz72mr7xb","exshwew2w2jv3n7r","hxrxgzvgms3incmf","hymu5oh2f5ctk5jr","jkisbjnul226jria","lag7djeljbjng6bu","o3l65o4qzcxs327j","qsk2jzo2zh523r24","t7k6g7fkndoggutd","xfllvjyax4inadxh","ygtjzi2wkfonj3z7","yycjajwpguyno4je"] 0