gl.recommender.util.compare_models not creating a plot

User 2568 | 5/9/2016, 3:56:45 AM

I'm following along with the Building a Recommender with Implicit Data - Million Song notebook and noticed that

result = gl.recommender.util.compare_models(test_set, [popularity_model, item_sim_model],
                                        user_sample=.1, skip_set=train_set)

did not create a plot as it does in the tutorial. I'm running 1.9

Comments

User 2506 | 5/9/2016, 5:57:22 PM

are you doing this on an ipython notebook?


User 2568 | 5/10/2016, 3:00:54 AM

I'm running this using 1. Jupyter notebook v 4.2 2. GraphLab 1.9 on an EC2 instance (i.e., remote) 3. I've set gl.canvas.set_target('ipynb') 4. I use Chrome browser

Either the functionality has changed in 1.9 OR its because i'm running GraphLab remotely.


User 2506 | 5/10/2016, 3:04:00 PM

Answered in another thread: http://forum.dato.com/discussion/632/allow-canvas-to-be-accessible-from-an-external-ip-address-port

The Canvas browser target is currently designed to run only on localhost for security reasons, and because the interaction between Python environment and Canvas UI is what powers the experience, so other remote users (who are not using the Python console) will not have the same experience. If you are running a Python console over a remote session (ssh) then you will have a problem with Canvas being on localhost like you describe. Exposing it over plain HTTP over the public internet would potentially leak data (the contents of your SFrames or other data structures) so it is not advisable.

However, there is a workaround you can try, which will allow a local (to your machine) web browser to access Canvas running on a remote server (over ssh).

You can find the port that Canvas is running on by calling: gl.canvas.gettarget().server.getport() Then, run an SSH tunnel to the remote server from your local machine, forwarding that port: ssh -L <port>:localhost:<port> remote.server Now you should be able to browse to that port on localhost (on your local machine) and get the Canvas experience locally, while having the traffic securely forwarded via SSH. You will need to leave the SSH session open while using Canvas for the port forwarding to continue. Make sure to browse to: http://localhost:<port>/index.html


User 2568 | 5/12/2016, 6:22:01 AM

I think the problem is with compare_models, not the remote use of canvas.

I used the SSH method to open a browser window to the canvas, however this still did not generate a plot. I confirmed the canvas is working by running:

popularity_model.show()
item_sim_model.show()

I got the message "Canvas is updated and available in a tab in the default browser".

I went to the canvas and I can see both models in the model tab. When I click on popularity_model I can view the table in the "Summary" tab and the chart in the "Evaluate" tab. WHen I click on COmparison I just get "Loading" and nothing happend.

When I run

result = gl.recommender.util.compare_models(test_set, [popularity_model, item_sim_model],
                                        user_sample=.1, skip_set=train_set) 

I don't see a plot in the notebook, I don't get a message and I don't see anything in the canvas. I also tried 'make_plot=True' to no avail. Have you tried running the notebook and checking that it works on your system in 1.9?

Given that I can do

gl.canvas.set_target('ipynb')
popularity_model.show(view='Evaluation')

and see the model evaluation in the notebook, why shouldn't this work with compare too?


User 2568 | 5/13/2016, 4:31:18 AM

I believe the following is the same bug, but when using show. Having set gl.canvas.set_target('ipynb')

similarity_model.evaluate(validate_set[:1000], verbose=False, cutoffs=[1, 2, 3, 4, 5]);
similarity_model.show(view='Evaluation')

plots a precision-recall curve in the notebook for these cutoffs. However

similarity_model.show(view='Comparison')

creates a blank area with the word loading, then nothing happens.


User 2506 | 5/16/2016, 9:25:24 PM

Hi, Sorry for the inconvenience.

compare_model plotting capability is hard to discover right now and we will definitely fix it

you can use the following to produce Precision-Recal plot :

graphlab.showcomparison(modelperformance,[popularitymodel, itemsim_model)


User 2568 | 5/16/2016, 9:33:10 PM

Thank, I try that later. If you look back at my original post, you will see that my comment was based on the Building a Recommender with Implicit Data - Million Song notebook/tutorial . I suggest you edit this so others don't have the same question/problem.