Tuning convolution neural net parameters

User 2051 | 6/13/2015, 5:35:11 PM

Hi,

My convolution neural net converged pre-maturely at iteration 1. I am using the default parameter setting (see below). Besides decreasing learning_rate, any other wisdom I can use to slow the learning? Thanks!!

network layers

layer[0]: ConvolutionLayer initrandom = gaussian padding = 0 stride = 2 numchannels = 10 numgroups = 1 kernelsize = 3 layer[1]: MaxPoolingLayer padding = 0 stride = 2 kernelsize = 3 layer[2]: FlattenLayer layer[3]: FullConnectionLayer initsigma = 0.01 initrandom = gaussian initbias = 0 numhiddenunits = 100 layer[4]: RectifiedLinearLayer numhiddenunits = 10 layer[5]: DropoutLayer threshold = 0.5 layer[6]: FullConnectionLayer initsigma = 0.01 initrandom = gaussian initbias = 0 numhidden_units = 5 layer[7]: SoftmaxLayer

end network layers

network parameters

learning_rate = 0.001 momentum = 0.9

end network parameters

Comments

User 19 | 6/14/2015, 8:45:52 PM

Hi Tina,

Can you also include the code you used to create the model, as well as the model progress summary? If you have tried decreasing the learning rate, what happens? How large is your dataset?

Looking forward to helping, Chris


User 2051 | 6/14/2015, 10:39:31 PM

Hi Chris,

Here is my code, most of them are from the tutorials.

import graphlab as gl import os train_sf = gl.image_analysis.load_images(WORKING_DIR, random_order=True) train_sf['image'] = gl.image_analysis.resize(train_sf['image'], 256, 256) train_sf['wnid'] = train_sf['path'].apply(lambda x: x.split('/')[-1].split('.')[-2]) train_sf.save(WORKING_DIR + '/sframe/train_shuffle') gl.canvas.set_target('ipynb') train = gl.SFrame(WORKING_DIR + '/sframe/train_shuffle') train.head() unique_labels = train['wnid'].unique().sort() class_map = {} labelframe = gl.SFrame.read_csv(WORKING_DIR +'/labels.csv') levels = labelframe['level'] for i in range(len(unique_labels)): class_map[unique_labels[i]] = levels[i] train['label'] = train['wnid'].apply(lambda x: class_map[x]) train.head() mean_image = train['image'].mean() training_data, validation_data = train.random_split(0.8) net = gl.deeplearning.create(training_data,'label') net.layers[4].num_hidden_units = 10 model = gl.neuralnet_classifier.create(training_data[['image','label']], 'label', network=net) pred = model.classify(training_data) pred eval_ = model.evaluate(training_data) eval_


User 1190 | 6/15/2015, 9:03:22 PM

Hi Tina,

What is the dataset you are using? How many images are there and how many unique labels?

What is the accuracy you get at the end?

Thanks, -jay