Replicating fitnet training performance with trainNetwork

3 views (last 30 days)
I am trying to design my own deep neural hidden network to approximate a functional relationship between an input vector and one target variable but I fail to obtain good results when designing my own deep neural network and I would like to understand the reason. The fit is pretty much perfect when I use the fitnet & train functions. To narrow down my problem, I brought it down to a one input, one target problem. The total number of samples is 2000.
The actual relationship looks as follows (input and output variables are already mapped to [-1,1]):
Capture4.JPG
Fitnet / train
Using fitnet, I achieve almost arbitrary accuracy using the following options to create a NN with two hidden layers with ten nodes each:
trainFcn = 'trainlm';
hiddenLayerSize = [10 10];
net = fitnet(hiddenLayerSize,trainFcn);
net.trainParam.min_grad=1e-10;
net.trainParam.max_fail = 500;
net.trainParam.epochs=5000;
net.divideFcn = 'dividerand';
net.divideMode = 'sample';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.performFcn = 'mse';
[net,tr] = train(net,x,t, 'useparallel', 'yes');
In this case, the training only takes a couple of seconds and the resultant fit is pretty much perfect:
Capture5.JPG
trainNetwork
However, I fail to obtain a good fit when designing my own deep neural network (I am using an imageInputLayer as suggested in another post)
layers = [imageInputLayer([1 1 1]);
fullyConnectedLayer(10)
tanhLayer
fullyConnectedLayer(10)
tanhLayer
fullyConnectedLayer(1);
regressionLayer];
options = trainingOptions('sgdm', 'maxepochs', 10000, 'minibatchsize', 2000, 'InitialLearnrate', 0.1,'Plots','training-progress');
XNew = reshape(x, [1,1,size(x,1),size(x,2)]); % size(XNew) = [1, 1, 1, 2000]
trainedNet = trainNetwork(XNew, t', layers, options);
I tried playing around with the Initial learning rate (from 0.1 to 1e-5), the minibatchsize (from 10 to all samples), the solver and the activation functions but after a fast decrease the RMSE always gets stuck around 0.1:
Capture2.JPG
and this is the fit I get:
Capture3.JPG
Am I doing anything fundametally wrong or does anyone have tips on how to improve the learning?
Thanks a lot in advance!
  3 Comments
Yifan Wei
Yifan Wei on 22 Jun 2021
I have faced same issue here. did you get a chance to look at why this happened?
ytzhak goussha
ytzhak goussha on 30 Jun 2021
You should try to run this without normalization in the imageInputLayer since you already scaled it [-1 1].
The defult is normalization be zero centering by subtracting the mean, the mean is a learnable parameter.
Try:
imageInputLayer([1 1 1],'Name','input','Normalization',"none")

Sign in to comment.

Answers (0)

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!