MATLAB Answers

Céldor
0

Mapminmax process function causes that NN incorrectly simulates outputs

Asked by Céldor
on 6 Apr 2016
Latest activity Commented on by Brendan Hamm on 12 Apr 2016
Hello,
I have a problem with some outputs from a trained custom neural network. I am using MATLAB 2014 with NN ToolBox ver 8.2.
I have created a simple feedforward NN for classification. I have used some Inputs and Targets, trained the NN, and tried to simulate Outputs given Inputs from the same range. The NN I created gives me incorrect outputs. First it returns outputs only with 0,1 range while the targets are in rage of -6 ... 3 and when I add a process function 'mapminimax' to input and output, the results are wrong: 1. when targets are 0 ... 1, there is an offset of 0.5 such that the outputs are 0.5 and 1 2. when targets are e.g. -6 ... 3, the outputs are around -3 ... 3
I am trying to understand what I am doing wrong.
PS. I have already asked this question in SO and also I provided some more details code: http://stackoverflow.com/questions/36449224/mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs
The code to test NN
clear all; close all; clc;
net = NNPatRec;
%net.inputs{1}.processFcns = {'mapminmax'};
%net.outputs{2}.processFcns = {'mapminmax'};
Inputs = -10:10;
%Targets = [-6*ones(1,11) 3*ones(1,10)];
Targets = [zeros(1,11) ones(1,10)];
[net,tr] = train(net,Inputs,Targets);
net(-10:10)
The code to create NN:
function net = NNPatternRecognition
net = nntest;
end
function net = nntest
net = network;
net.numInputs = 1;
net.numLayers = 2;
net.biasConnect = [1 1]';
net.inputConnect = [1; 0];
net.layerConnect = [0 0; 1 0];
net.outputConnect = [0 1];
% Inputs
%net.inputs{1}.processFcns = {'mapminmax'};
net.inputWeights{1}.learnFcn = 'learngdm';
% layers 1 (at input)
net.layers{1}.initFcn = 'initnw';
net.layers{1}.netInputFcn = 'netsum';
net.layers{1}.transferFcn = 'tansig';
net.layers{1}.size = 3;
% layers 2 (hidden)
net.layers{2}.initFcn = 'initnw';
net.layers{2}.netInputFcn = 'netsum';
net.layers{2}.transferFcn = 'purelin';
net.layers{2}.size = 1;
% Network functions
net.adaptFcn = 'adaptwb';
net.derivFcn = 'defaultderiv';
net.divideFcn = 'dividerand'; %'divideblock';
net.initFcn = 'initlay';
net.performFcn = 'crossentropy';
net.trainFcn = 'trainscg';
% Outputs
%net.outputs{2}.processFcns = {'mapminmax'};
%net.outputs{2}.exampleOutput = [0 1];
net.trainParam.showWindow = false;
net.trainParam.showCommandLine = true;
end

  0 Comments

Sign in to comment.

2 Answers

Answer by Greg Heath
on 8 Apr 2016
 Accepted Answer

close all, clear all, clc, plt=0, tic
x = -10:10; N = length(x)
trueind = 1 + [zeros(1,11) ones(1,10)];
t = full(ind2vec(trueind))
plt = plt+1, figure(plt), hold on
plot( x( 1:11), trueind( 1:11) ,'o' )
plot( x(12:21), trueind(12:21),'ro' )
axis([ -11 11 0 3 ])
title('CLASS INDICES')
rng('default')
net = patternnet;
[ net tr y e ] = train( net, x, t );
outind = vec2ind(y)
plot( x( 1:11), outind( 1:11) ,'x' ,'LineWidth',2)
plot( x(12:21), outind(12:21),'rx' ,'LineWidth',2)
err = outind~=trueind;
Nerr = sum(err) % 1
PctErr = 100*Nerr/N % 4.7619
Hope this helps.
For details, remove the semicolon to get
net = net
ALSO, for a trn/val/tst breakdown use
tr = tr
Hope this helps.
Thank you for formally accepting my answer
Greg

  3 Comments

Hi Greg,
Thanks for your answer. I followed your code.
In the final matrix produced by Brendan and yourself, there are three categories not two as I used in my example. Is it a standard routine in classification to always add one extra category?
Thanks
I don't know where you got that idea. Did you run my program?
There should be exactly 2 rows in the output. The probability of Class_1 and the probability of Class_2. You just refer to these as Class_0 and Class_1 respectively.

Sign in to comment.


Answer by Brendan Hamm on 6 Apr 2016

You would likely have better luck if you just started with the patternnet which is meant for NN classification.
What I see that is wrong with your current implementation is you have a linear transferFcn for your second layer. For classification purposes this should really be a softmax function. That is change
net.layers{2}.transferFcn = 'purelin';
to
net.layers{2}.transferFcn = 'softmax';
There are also 2 functions used for the processFcns for the the input and output:
net.outputs{2}.processFcns == {'removeconstantrows', mapminmax};
net.inputs{1}.processFcns == {'removeconstantrows', mapminmax};

  4 Comments

Show 1 older comment
mapminmax will map the output values to a minimum and maximum value. By default this is set to the range [-1 1]. So, I suppose I missed this part above, we would need to change the range to be [0,1]. This is controlled in the processParams:
p.outputs{2}.processParams{1}.max_range = 0; % params for remove constant rows
p.outputs{2}.processParams{2}.ymin = 0; % Params for mapminmax
p.outputs{2}.processParams{2}.ymax = 1; % Params for mapminmax
Furthermore with a classification problem the output should really be a probability of belonging to each class. For this reason for a binary classification you would expect 2 outputs for every sample, the probability of class1 and the probability of class2.
So, You really want your targets to be a matrix of ones and zeros where 1 indicates belonging to that class (i.e. you know it belongs to class1 so the probability is 1). You can get this result by calling:
dummyvar(Targets+1)';
I use the +1 as dummyvar expects all inputs to be positive integers (or alternatively categorical or strings) So all in all this could have been accomplished with the following:
Inputs = -10:10;
Targets = [ones(1,11) 2*ones(1,10)]; % I'll Increment by 1 here (Class name is irrelevant)
p = patternnet(10); % For classification problems
Targets = dummyvar(Targets)'; % Make this into an indicator matrix
p = train(p,Inputs,Targets); % train the NN
Tpred = p(Inputs) % Verify the results with our training set.
Tpred =
Columns 1 through 3
1.0000 1.0000 1.0000
0.0000 0.0000 0.0000
Columns 4 through 6
1.0000 1.0000 1.0000
0.0000 0.0000 0.0000
Columns 7 through 9
1.0000 1.0000 1.0000
0.0000 0.0000 0.0000
Columns 10 through 12
1.0000 1.0000 0.0000
0.0000 0.0000 1.0000
Columns 13 through 15
0.0000 0.0000 0.0000
1.0000 1.0000 1.0000
Columns 16 through 18
0.0000 0.0000 0.0000
1.0000 1.0000 1.0000
Columns 19 through 21
0.0000 0.0000 0.0000
1.0000 1.0000 1.0000
As a shameless plug, if you ever want your Machine Learning Algorithms to feel a little bit less "black-box", consider attending the Machine Learning with MATLAB course.
Many thanks for this Brendan. Everything's working :)
Regarding mapminmax, I also found what I needed to set yesterday. I had to go through the process of how the command patternnet creates a a netowrk and realised it uses feedforwardnet. But somewhere in the code, there's a line where the ymin is set to 0. I have set ymin by myself and everything started to work :)
And thanks for the comment on preparation of Targets for classification. This is another thing I had to go through by myself and realised that NN works much better if there are multiple outputs, each representing individual class.
Thanks :)
PS. I have just learned a new command: dummyvar

Sign in to comment.