Feature Request: benchmark!

User 1916 | 5/12/2015, 3:41:39 PM

Hi, I installed G-Create on a variety of Macs and Linux (the GPU version on Linux) machines that each had its own tweaks and customizations, as well as somewhat different CUDA versions, and I could use an out-of-the-box test/benchmark function that would alert me early of configuration problems. At least one "benchmark" section has been removed from your docs, while the academic papers and benchmarks linked are too involved for quick tests. Obviously it would be out of the question to include MBs and GBs of test data with Graphlab, but some scripts would go a long way, I think, to help your new users with their setups, I certainly have spent a couple of hours looking for suitable examples but it looks like it is not so simple, partly because of older tutorials pointing to missing Internet resources.

Not quite worthy of a post on its own, there seems to be some typo or inconsistency in your "welcome message" when I start the 1.3 GPU server, it says something like

">>> import graphlab as gl A newer version of GraphLab Create (v1.3) is available! Your current version is v1.3.0.gpu. New features in 1.4:

Perhaps you turned on a flag somewhere anticipating the 1.4 release.


User 1394 | 5/12/2015, 10:11:27 PM

Hey Anastasios -

Thanks for reaching out and trying GraphLab Create. Can you describe a couple sample benchmarks that would be relevant to you? What type of configuration problem are you encountering/concerned about?

Have you taken a look at the Learn section on dato.com? Specifically, check out the Gallery (http://dato.com/learn/gallery/) - there are a few examples specifically for GPU tasks that might be relevant.

We just released GraphLab Create 1.4 so I will dig into the message you are seeing on the GPU installation.



User 1916 | 5/15/2015, 12:31:48 PM

Well, I felt that the CUDA part would be especially flakey, not so much on your side but in the grand scheme of things, after all Nvidia driver releases do come with health warnings for CUDA features. Considering everything I've done quite well on my 32GB RAM, 4GB VRAM machine, one warning and no crashes, but a relatively demanding NN test with "known" results would clinch it for me.

Perhaps I guessed correctly about that 1.4 release, I just don't see any info on the 1.4gpu one!