Update graphlab.recommender.MatrixFactorizationModel with new training data

User 379 | 6/24/2014, 7:11:37 PM

Is it possible to update graphlab.recommender.MatrixFactorizationModel with new training data? Also, is it possible to set initial values when creating a graphlab.recommender.MatrixFactorizationModel?


User 18 | 6/25/2014, 5:42:14 PM

Hi there,

Currently, no to both questions. Models are immutable, just like the basic data structures (SFrame and SGraph). So to update a model means creating a new model. From a machine learning perspective, warm start may or may not help with faster convergence on the new dataset. (At least I've seen both behaviors, though not specifically for matrix factorization.)

But we can provide syntactic API sugar to make the call easier. Are you looking for custom initialization mainly as a way to get fast model updates? Or for something else? Would something like the following work for you?

newmodel = oldmodel.update(newobservationdata, ...)


User 379 | 6/25/2014, 7:37:53 PM

Hi Alice,

Exactly. I think previous graphlab API (the open source one) and GraphChi have the ability to import previous model and use it as initial values. (--loadfactorsfrom_file option).

Thanks, Milad

User 18 | 6/25/2014, 8:15:38 PM

Ok, thanks for the clarification. We'll include model updating in our next round of planning discussions.

User 379 | 6/25/2014, 8:19:52 PM

Hi Alice,

Thanks for your quick response and consideration.

User 5202 | 5/15/2016, 6:41:17 PM

Hi, I wanted to follow up on the status of this request. Specifically I'm looking to leverage the FM recommender framework to update data in batch, e.g. feeding in new data on an hourly basis WITHOUT having to retrain. Additionally, as new user/item features are presented having the various model parameters update appropriately. If this is not possible with the code as is, could these necessary values (e.g. gradient and learning rates) be exposed for customization? Thanks!

User 5202 | 5/15/2016, 8:55:26 PM

I can see that this would handle updates the way I described above. If there are new user/item features will this create any issues?

User 1207 | 5/16/2016, 7:37:08 PM

Hello @dvsstats,

Currently, the factorization models don't support adjusting the user and item factors very well after the model is trained. The underlying models that we use (non-convex factorization models with sampling of negative examples) tend to be very, very good for building recommenders, but unfortunately, the time that would be required to properly adjust the models in the presence of new data would be almost as much as retraining the entire model. We're hoping to support your use case better in the future, but it's proven to be a bit of a technical challenge. Currently, it mainly uses this information to exclude predictions; but if you have a lot of new users and items coming in, the item_similarity model tends to be much better at handling this situation.

Hope that helps! -- Hoyt

User 5202 | 5/16/2016, 8:08:24 PM

Thank you for the prompt reply @hoytak! So, coming back to my question, is there any way to gain access to the gradient/learning rate information? I've built a similar system in R where the various gradient/learning rates (using an FTRL optimization scheme) dynamically adapt as new data comes in, but was hoping to avoid recreating this in Python. It seems that retraining hourly when there's a fairly large amount data (where context over users/items changes somewhat frequently) would be prohibitive. What is the recommended approach in this setting using GraphLab?

User 1207 | 5/16/2016, 8:48:27 PM

Hello @dvsstats,

I'd say you should try retraining the model from scratch each hour and see if it's fast enough. It's one of the fastest ones out there, so you may be pleasantly surprised.

As for the gradients: you do have access to all of the factors in a trained model through m.get("coefficients"), and from that you can compute the gradients. The gradients in the default version use sgd or adagrad, so the full gradients are never computed.

Hope that helps! If a full retraining is prohibitive, then I'd like to hear more about your use case and how we can help make it work.

-- Hoyt