In order to investigate the claim of Jeffrey Dean about the graphlab W.R.T structured graphs in his paper, I coded a small deep neural network (10 hidden layers with each layer having 10 nodes). I used Contrastive Divergence (CD) algorithm for learning the structure in data. I am facing problem in Scatter phase, however I think I may resolve that. I implemented Run time scheduling feature to communicate between the layers. One of the problems I am facing is the parallelism across the data set. I mean if there are 1 million entries, it would be good if I could be able to divide the dataset into 10 different sets and I can pass each dataset to typical graphlab engine and computing each engine in parallel. Is it possible to do so?
One more feedback I would like to share is how about signalling a layer instead of signalling a particular vertex in the layer? After the structure of graph is created, I am loading the graph for every dataset from the hard disk and then re-computing the structure of graph and then writing back to the disk so that next data set could pick up the changed graph. It seems there is lot of I/O in this process. Is there any method to by-pass this IO?
It would be great if you feed back about the changes that could be made to our Graphlab to suit structured graphs model. It would be really interesting for me to explore this problem in depth.