Saving multiple classifier

User 1273 | 2/24/2015, 3:21:13 PM

Hello, is there a way to save multiple models in a single file? Let's say I have three trained models and I want to save them in a single file and reload them later, is it possible? Even if by tweaking with the code (e.g. "reading" the models' structures and saving them in a .txt, and rebuilt it later by code)? In the latter case please be specific on what to do...

Is it possible to even save a single model to a single file (right now it is saved to a directory with various files in it)?

Also, saved models are very heavy: e.g. I have a 15 Mb model for a 30 Mb dataset. Why is that? If I use a 30 Gb dataset will I end up with a 15 Gb model? That would be completely unpractical to handle. Is there a way to reduce the saved model dimension?

Comments

User 91 | 2/24/2015, 4:44:34 PM

You can use the Graphlab pickler (released in GLC 1.3) to do so.

    from graphlab.util import gl_pickle
    import graphlab as gl

    obj = {'model-1': model_1,
               'model-2': model_2,
               'data: gl.SArray([1,3,4])}

    # Setup the GLC pickler
    pickler = gl_pickle.GLPickler(filename = 'foo-bar')
    pickler.dump(obj)

    # The pickler has to be closed to make sure the files get closed.
    pickler.close()

If you do have boosted trees as your model and a lot of classes and trees, the size can be a few megabytes. However, the model size should not depend on your data size. So even if you have a 30GB dataset, it should not be more than 15mb.


User 481 | 3/26/2015, 12:30:09 AM

Even a 15mb model is large to transfer over wire. Is there any future provision to make model json serializable? I get a "can't serialize UnityModel" error


User 91 | 3/26/2015, 12:35:17 AM

Our file format is native and binary and incredibly efficient. JSON is not the most efficient format so 15MB in the binary format would be a lot more in JSON (non trivially).

Could I understand your use case a bit better? If you are interested in querying the model via JSON, then take a look at Dato predictive services which provides a nice API surface layer for <100ms query latency of complex models (including multiple classifiers).


User 481 | 3/27/2015, 11:39:08 PM

My application needs to potentially serialize the model to be able to http "put" it to another server. So something like a string representation would be helpful. What I get using a gl_pickle is a file which is not exactly ideal to http "put". Does that make sense? Any way I can do that?


User 398 | 3/28/2015, 12:04:14 AM

Hi there,

Thanks for your question. As Krishna pointed out, we cannot serialize our models to JSON. Are you sure you need to pass the entire model as the body of a PUT request? In Dato Predictive Services, we store models in their native binary format in S3, load the model on the server side, and then can make predictions once the model is loaded. Would that kind of approach work for your use case?

Best, Robert


User 4 | 3/28/2015, 12:17:11 AM

Hi @MSH,

I believe HTTP in general allows binary data as a PUT message body. Does the service you are sending a PUT request to require the message to be valid JSON? If so, you could wrap our binary (pickled) model as JSON using base64 encoding. Let me know if this fits your requirements and I can try to provide some sample code to do this.


User 481 | 3/30/2015, 10:25:06 PM

Hi @Zach Thanks for the reply.

We have a peculiar case I suppose where we cannot "save" the model on a disc/db because it necessitates dedicated ops resources. So the plan is to compute a model, serialize it and write it to an external "entity" service over http put. Imagine the entity service as a persistent object store accessible through a http rest api.

When a request for a recommendation arrives, the model is retrieved from this "entity" service and used to generate the recommendation, and the recommendation is served back to the requesting app and maybe saved in the entity service itself. The reason to do this, is to avoid writing a bunch of recommendations back to the "entity" service. For example, if 10 recommendations are generated for 400K users, thats 4M recommendations. But if only 1K users arrive requiring only 10K recommendations. This means 4M-10K recommendations written to entity service are overhead.

Http may allow binary data in PUT requests but the model size is around 1-10mb in my experience as a file. Retriving this sized data on the fly is not possible. Most cacheing tools require {key:value} serialization which cannot be done, so model has to be retrieved for all requests.

I hope I am making sense. The code for wrapping pickled model as JSON using base64 encoding would help. Any comments on the set up? Also storing models on s3 is probably not an option because we are committed to using ibm bluemix.

Thanks Manas


User 4 | 3/30/2015, 10:32:04 PM

Hi @MSH,

I will be happy to experiment to try to find the right code for serializing and deserializing a model to JSON with base64, but it may not help since this will not reduce the file size of the model (in fact it will increase, as base64 has some overhead). It sounds like serializing the whole model on each request is too much overhead for this application. In your scenario, it totally makes sense to host the model somewhere and query it in response to requests, rather than pre-generate all the (many possible) predictions into a DB.

We do have another product you may be interested in. Dato Predictive Services is a managed, load-balanced hosted model as a REST API. We provide the tools to manage these hosted models as part of GraphLab Create, and deploy your own models to them. Perhaps this will fit your needs?


User 481 | 3/30/2015, 10:37:59 PM

@Zach Quick question, by "host a model somwhere" you mean amazon s3?


User 4 | 3/30/2015, 11:09:34 PM

@MSH: In Predictive Services, the model is hosted as part of a running service (I believe it is also copied to S3 so that new machines in the cluster can spin up and get a copy of the model, but for the most part it stays local to the machines running the service, so that REST calls are quick and do not require copying the model). The service is controlled via the Python API in GraphLab Create (to create/update/terminate/monitor the service). The service itself runs on one or more machines, behind a load balancer (the number of machines you need will depend on traffic patterns). Currently, these machines must be on EC2 (and the GraphLab Create API manages those EC2 machines) but in the future we will support other architectures/PaaS. Does that answer your question?


User 481 | 3/30/2015, 11:55:05 PM

@Zach It does. Maybe predictive services are better suited for our use case. Does the licensing for predictive services is different from graphlab-create? Meaning, do we have to get different licesnes for predictive services and graphlab create?


User 1394 | 3/31/2015, 8:08:37 PM

Hey MSH -

Yes and no. Yes, Dato Predictive Services has the same licensing terms as GraphLab Create (meaning the terms & conditions are the same across all Dato products). However, Dato Predictive Services require separate software licenses from GraphLab Create.

Let's talk offline about IBM Bluemix and Dato Predictive Services. Ping me at rajat@dato.com so we can talk more through this use case.

Thanks,

Rajat


User 5191 | 5/20/2016, 9:49:29 PM

Having trouble finding the Graphlab pickler in the most recent release of Graphlab. Am I missing something here?

` import graphlab as gl from graphlab.util import gl_pickle


ImportError Traceback (most recent call last) <ipython-input-247-62ea325051c0> in <module>() 23 import sys 24 import pickle ---> 25 from graphlab.util import gl_pickle

ImportError: cannot import name gl_pickle `