Why is predicting the output of a trained neural network by using "net/sim" different than manually calculating the output using the network weights, biases and transfer functions?

Why is predicting the output of a trained neural network by using "net/sim" different than manually calculating the output using the network weights, biases and transfer functions?

Refer to the following example:

%% create random data
rng(1);
xdata = rand(5,100);
ydata = rand(1,100);

%% fit a forward network
nlayersize = 10;
net = feedforwardnet(nlayersize);
net.trainParam.showWindow = false;
net = train(net,xdata,ydata);
net.layers{1}.transferFcn

%% calculate the model output at a new point
xNew = rand(5,1);
y1 = net(xNew); % can use sim as well

%% manually calculate model output
IW = net.IW{1};
LW = net.LW{2,1};
b = net.b;

y2 = IW*xNew+b{1};
y2 = tansig(y2);

y2 = LW*y2+b{2};

dy = y1-y2

"dy" is not zero but it should be.

Answer 1

[MathWorks Support Team]

The difference between the network predictions using "net/sim" functions and manually computing the prediction using the code in the question is in the "minmax" processing that most neural networks automatically apply to the input and target (output) data. You can refer to the following documentation link for the "mapminmax" function which is used for this pre-processing of the training data:

https://www.mathworks.com/help/deeplearning/ref/mapminmax.html

The settings for this pre-processing steps are stored in "net.inputs{1}.processSettings{1}" for the input pre-processing and "net.outputs{2}.processSettings{1}" for the output pre-processing. So, if you change the code to the following you should obtain a ‘dy’ difference of practically zero (something of the order of 10^-16):

xNew = rand(5,1);
y1 = net(xNew);

xNew = mapminmax('apply', xNew , net.inputs{1}.processSettings{1}); % apply input pre-processing

IW = net.IW{1};
LW = net.LW{2, 1};
b = net.b;

y2 = IW*xNew+b{1};
y2 = tansig(y2);

y2 = LW*y2+b{2};
y2 = mapminmax('reverse', y2, net.outputs{2}.processSettings{1}); % reverse the output pre-processing

dy = y1 - y2