Real time or post session learning curve display

User 1975 | 5/27/2015, 10:40:47 PM

Recently I'm working with convolution neural networks. Please, give me an explanation how to create learning curve in real time during the learning process or post learning. I can't find any simple way to aquire the then-current score data. I found a complicated way, namly saving current model with "modelcheckpointpath" option after each iteration and using some multiprocessing option make a predict and redirect this prediction score to an SGraph instance along the learning process. Please, give me a clarification or advice. Any sample code to this issue and to multiprocessing(parallellization) options would be higly appreciated. Thank you in advance,

Comments

User 91 | 5/30/2015, 6:51:07 PM

Unfortunately, currently there isn't a way to get the learning curve in real time because the training process is a blocking call. Your only option is to do what you have described i.e load the model from modelcheckpintpath and every x-mins (in a separate process) and plot the statistics that you have.


User 1975 | 5/31/2015, 11:51:22 AM

Thank you, but meanwhile I wrote on the deprecated google.groups: Joseph Gonzalez 2013. 05. 27. Alternatively, if you use the synchronous engine we have support for periodic checkpoints. However, the convergence properties of the collapsed Gibbs sampler are not well understood in the synchronous setting. Could you kindly give a brief explanaton about this method?


User 1975 | 5/31/2015, 12:01:01 PM

Alternatively is there any way, function to reach the currently running processes' parameters from an other parallel application?


User 1975 | 5/31/2015, 12:06:04 PM

Sorry, but fo course I red not wrote :) : Thank you, but meanwhile I wrote on the deprecated google.groups: Joseph Gonzalez 2013. 05. 27. Alternatively, if you use the synchronous engine we have support for periodic checkpoints. However, the convergence properties of the collapsed Gibbs sampler are not well understood in the synchronous setting. Could you kindly give a brief explanaton about this method?