Can not launch predictive service

User 1228 | 10/28/2015, 9:01:50 AM

Hi there. I want to use the predictive service but I get an error when deploying.

I do:

python env = gl.deploy.Ec2Config(region='eu-central-1', instancetype='m3.xlarge', awsaccesskeyid='xxxxx', awssecretaccesskey='xxxxxx', securitygroup='xxxxx', subnet_id='xxxxx')

ps = gl.deploy.predictiveservice.create('sample-service-onefive', env, 's3://mybucket/name')

The user is valid and has RW access to the bucket. The region is valid as well. But I always get the following error.

` [INFO] Error hit while uploading to S3: AWS S3 operation failed [INFO] Retrying 1 out of 5 [INFO] Error hit while uploading to S3: AWS S3 operation failed [INFO] Retrying 2 out of 5 [INFO] Error hit while uploading to S3: AWS S3 operation failed [INFO] Retrying 3 out of 5 [INFO] Error hit while uploading to S3: AWS S3 operation failed [INFO] Retrying 4 out of 5 [INFO] Error hit while uploading to S3: AWS S3 operation failed [INFO] Retrying 5 out of 5

RuntimeError Traceback (most recent call last) <ipython-input-41-df6e03229ad8> in <module>() ----> 1 ps = gl.deploy.predictive_service.create('sample-service-onefive', env, 's3://tda-graphlab-logs/logs')

/usr/local/lib/python2.7/site-packages/graphlab/deploy/predictiveservice.pyc in create(name, ec2config, statepath, numhosts, description, apikey, adminkey, sslcredentials, corsorigin, port) 203 try: 204 fileutil.uploadtos3(__defaultconfigpath, statepath + "/license", --> 205 awscredentials = awscredentials, silent = True) 206 region = fileutil.gets3bucketregion(s3bucketname, awscredentials) 207 except S3ResponseError as e:

/usr/local/lib/python2.7/site-packages/graphlab/util/fileutil.pyc in uploadtos3(localpath, s3path, isdir, awscredentials, silent) 269 logger.info("Retrying %s out of %s" % (numretries, RETRYTIMES)) 270 if numretries == RETRYTIMES: --> 271 raise e 272 time.sleep(__SLEEPSECONDSBETWEENRETRIES) 273

RuntimeError: AWS S3 operation failed `

Comments

User 2506 | 10/28/2015, 11:26:45 PM

Hi Marcel,

Can I check with you whether you are trying to launch PS into a private VPC?

If so, does your subnet have internet connectivity?

For debugging, can you try running with this Config instead?

env = gl.deploy.Ec2Config(region='eu-central-1', instancetype='m3.xlarge', awsaccesskeyid='xxxxx', awssecretaccess_key='xxxxxx')


User 1228 | 10/29/2015, 7:44:58 AM

Hi Shaowei

Thanks for your quick answer. I did what you asked for and got the following: `python env = gl.deploy.Ec2Config(region='eu-central-1', instancetype='m3.xlarge', awsaccesskeyid='xxxxx', awssecretaccesskey='xxxxxx') ps = gl.deploy.predictiveservice.create('sample-service-onefive', env, 's3://my_bucket/name')

RuntimeError Traceback (most recent call last) <ipython-input-6-1f7160c5ead3> in <module>() ----> 1 ps = gl.deploy.predictive_service.create('test-service', env, 's3://tda-graphlab-logs')

/usr/local/lib/python2.7/site-packages/graphlab/deploy/predictiveservice.pyc in create(name, ec2config, statepath, numhosts, description, apikey, adminkey, sslcredentials, corsorigin, port) 203 try: 204 fileutil.uploadtos3(__defaultconfigpath, statepath + "/license", --> 205 awscredentials = awscredentials, silent = True) 206 region = fileutil.gets3bucketregion(s3bucketname, awscredentials) 207 except S3ResponseError as e:

/usr/local/lib/python2.7/site-packages/graphlab/util/fileutil.pyc in uploadtos3(localpath, s3path, isdir, awscredentials, silent) 269 logger.info("Retrying %s out of %s" % (numretries, RETRYTIMES)) 270 if numretries == RETRYTIMES: --> 271 raise e 272 time.sleep(__SLEEPSECONDSBETWEENRETRIES) 273

RuntimeError: AWS S3 operation failed ` So the error looks the same.


User 1228 | 10/29/2015, 2:44:43 PM

OK. I realised that the region is not valid anyway. Should be 'eu-west-1'. But the error persists.


User 2506 | 10/29/2015, 3:00:28 PM

Thanks Marcel.

Can you help me by running 'pip list' and copy and paste the output here?


User 1178 | 10/29/2015, 5:23:25 PM

Hi Marcel,

Seems like you have trouble writing to S3, let's try the following:

  1. Let's see if you have the right environment setup to access s3, try the following in the console where you run the python:

      aws s3 ls s3://my_bucket/name'
  2. If above command succeeds, then try a few read and write operations on the s3 bucket and see if you have the right permission using the "aws s3" command

  3. If above command failed, then depend on what error you get, we may resolve differently. If it is a permission issue, then you may want to fix that. If it is something related to SSL, then you may want to be able resolved by either downgrading your awscli version or certifi version (see this post

Thanks! Ping


User 1228 | 11/2/2015, 10:15:51 AM

OK. we have some progress :-). Now we can launch an ec2 instance. However, we see now that the user must have the permission to do CreateLoadBalancer. Thats ok. Yet we want to fully understand all the permissions needed to use predictive services on ec2. Do you have something like a IAM policy to better understand the full chain? For us its a very important point since we deal with data from our clients. Thanks a lot.


User 1178 | 11/3/2015, 11:06:50 PM

Hi Marcel,

Thank you for your feedback. We are in the process of creating that and we will publish as part of our user guide in the coming release.

Ping


User 1228 | 11/20/2015, 11:43:23 AM

Hi Ping Thanks. Great.