Output of neural network is offset and scaled.. help!?

I am trying to simulate the outputs for a neural network myself for later translation to java so i can run it on a mobile device. For this i generated the following simulation code for a network with two hidden layers and tangent-sigmoid nonlinear function at all layers:

function [ Results ] = sim_net( net, input )

    y1 = tansig(net.IW{1} * input + net.b{1});

    y2 = tansig(net.LW{2} * y1 + net.b{2});

    Results = tansig(net.LW{6} * y2 + net.b{3});
end

The sim_net function is then held up against matlab's own functions using the following code:

clc

clear all

net = feedforwardnet([20 20]);

net.divideParam.trainRatio = 75/100; % Adjust as desired

net.divideParam.valRatio = 15/100; % Adjust as desired

net.divideParam.testRatio = 10/100; % Adjust as desired

net.inputs{1}.processFcns = {}; % no preprocessing

net.outputs{2}.processFcns = {};

net.layers{1}.transferFcn = 'tansig';

net.layers{2}.transferFcn = 'tansig';

net.layers{3}.transferFcn = 'tansig';

% Train and Apply Network

[x,t] = simplefit_dataset;

[net,tr] = train(net,x,t);

for i=1:length(x)

disp(i) = sim_net(net,x(i));

disp2(i) = sim(net,x(i));

end

plot(disp)

hold on

plot(disp2)

legend('our code','matlabs code')

the plot of the two outputs:

however, a quick inspection using the following edit reveals that matlabs results are offset by a factor of 5, and scaled by a factor of 5 also

plot(disp)

hold on

plot((disp2-5)/5+0.1)

legend('our code','matlabs code')

However, matlab's net function shouldn't even be able to give values above 1 when using tansig as the last activation function?

[Greg Heath]

One hidden layer is sufficient since it is also a universal approximator.

The fewer weights that are used, the more robust the design.

Answer 1

[Greg Heath]

The function input should be weights, instead of the net.

Is LW{6} a typo?

One hidden layer is sufficient since it is a universal approximator.

Since the function is smooth with four local extrema, only four hidden nodes are necessary.

The fewer weights that are used, the more robust the design. Also, this will make your java coding much easier.

Please use the notation h to denote the output from a hidden layer.

Your output can have any scale because of the default normalization/denormalization used within train

Code should be faster if you use dummy variables like IW = net.IW{1,1}, b1 = , b2 = , LW = ..

Thank you for formally accepting my answer

Greg

Answer 2

[Søren Jensen]

Thank you for taking the time to look through my code!

first, the training data is just an example. when i export it to java i need to use another network and more complicated data, so it really needs to work with two layers!!

second, net.LW{6} is not a typo (i can see how it would look that way). However, matlab has a weird way of organizing the layer and input weights:

BUT the problem is now solved, and thank you for pointing out matlabs normalization/denormalization. somehow there was a factor in the neural net that i did't reproduce in my sim_net function.

To anyone who might be stuck on this problem later here are the lines that solved it:

[xn,xs] = mapminmax(x);

[tn,ts] = mapminmax(t);

[net,tr] = train(net,xn,tn);

thereby the normalization is not run during training, and i avoid getting the weird scaling factor I don't know how to access in my network.

thanks very much to Greg Heath for his answer!