How to reproduce exact results of a patternnet network using feedforwardnet network.
4 views (last 30 days)
Show older comments
George Giasemidis
on 20 Oct 2015
Answered: Greg Heath
on 21 Oct 2015
I am trying to reproduce the results of a neural network created by "patternnet" function, using the "feedforwardnet" function instead. I fix
- the architecture
- the initialization method, and
- the seed in the randStream (for generating random numbers)
to be the same. However I still can't get identical results. Although I use the same initialization function I found that the difference is in the initial weights and biases. why are the weights and biases initialized differently? Debugging the functions I found that the "init" function is called more times than expected using the "patternnet" function. Why is this?
I post some code and results to prove that random number generator is not an issue. (A similar question has been asked here, but the problem has not been resolved).
TrainFeatNN = SelectedFeat40.Train;
TrainObsNN = trainToReduce.ObsClasses(:,2:3);
testInd = false(size(SelectedFeat40.Train,1),1);
testInd2 = false(size(SelectedFeat40.Train,1),1);
testInd3 = false(size(SelectedFeat40.Train,1),1);
testInd4 = false(size(SelectedFeat40.Train,1),1);
posclass = 1;
x = TrainFeatNN';
t = TrainObsNN';
%
%
%%Neural Networks patternnet #2
%
%
RandStream.setGlobalStream (RandStream ('mrg32k3a','Seed', 1234));
trainFcn = 'trainscg';
hiddenLayerSize = 10;
net = patternnet(hiddenLayerSize, trainFcn);
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','fixunknowns'};
net.divideFcn = 'divideblock';
net.divideMode = 'sample';
net.divideParam.trainRatio = 60/100;
net.divideParam.valRatio = 20/100;
net.divideParam.testRatio = 20/100;
net.trainParam.max_fail = 6;
net.trainParam.epochs = 1000;
net.performFcn = 'crossentropy';
[net,tr] = train(net,x,t);
y = sim(net,x);
PredProb_NN = transpose(y(2,tr.testInd));
testInd(tr.testInd) = true;
[XNN,YNN,~,AUC_NN] = perfcurve(TrainObsNN(testInd,end),PredProb_NN, posclass);
display(sprintf('Method: patternnet (#1): \nBest Epoch: %d,\nBest performance: %.4f,\nAUC: %.4f\n',tr.best_epoch,tr.best_perf,AUC_NN)
%
%
%%Neural Networks feedforwardnet #1
%
%
RandStream.setGlobalStream (RandStream ('mrg32k3a','Seed', 1234));
trainFcn = 'trainscg';
hiddenLayerSize = 10;
net2 = feedforwardnet(hiddenLayerSize,trainFcn);
net2.performFcn = 'crossentropy';
net2.outputs{numel(hiddenLayerSize)+1}.processParams{2}.ymin = 0;
net2.layers{net2.numLayers}.transferFcn = 'softmax';
net2.plotFcns = {'plotperform','plottrainstate','ploterrhist','plotconfusion','plotroc'};
net2.initFcn='initlay';
net2.layers{1}.initFcn='initnw';
net2.layers{2}.initFcn='initnw';
net2.layers{1}.transferFcn = 'tansig';
net2 = init(net2);
net2.input.processFcns = {'removeconstantrows','mapminmax'};
net2.output.processFcns = {'removeconstantrows','fixunknowns'};
net2.divideFcn = 'divideblock';
net2.divideMode = 'sample';
net2.divideParam.trainRatio = 60/100;
net2.divideParam.valRatio = 20/100;
net2.divideParam.testRatio = 20/100;
net2.trainParam.max_fail = 6;
net2.trainParam.epochs = 1000;
[net2,tr2] = train(net2,x,t);
y2 = sim(net2,x);
PredProb_NN2 = transpose(y2(2,tr2.testInd));
testInd2(tr2.testInd) = true;
[XNN2,YNN2,~,AUC_NN2] = perfcurve(TrainObsNN(testInd2,end), PredProb_NN2, posclass);
display(sprintf('Method: feedforwardnet (#1): \nBest Epoch: %d,\nBest performance: %.4f,\nAUC: %.4f\n',tr2.best_epoch,tr2.best_perf,AUC_NN2))
%
%
%%Neural Networks patternnet #2
%
%
RandStream.setGlobalStream (RandStream ('mrg32k3a','Seed', 1234));
trainFcn = 'trainscg';
hiddenLayerSize = 10;
net3 = patternnet(hiddenLayerSize, trainFcn);
net3.input.processFcns = {'removeconstantrows','mapminmax'};
net3.output.processFcns = {'removeconstantrows','fixunknowns'};
net3.divideFcn = 'divideblock';
net3.divideMode = 'sample';
net3.divideParam.trainRatio = 60/100;
net3.divideParam.valRatio = 20/100;
net3.divideParam.testRatio = 20/100;
net3.trainParam.max_fail = 6;
net3.trainParam.epochs = 1000;
net3.performFcn = 'crossentropy';
[net3,tr3] = train(net3,x,t);
y3 = sim(net3,x);
PredProb_NN3 = transpose(y3(2,tr3.testInd));
testInd3(tr3.testInd) = true;
[XNN3,YNN3,~,AUC_NN3] = perfcurve(TrainObsNN(testInd3,end), PredProb_NN3, posclass);
display(sprintf('Method: patternnet (#2): \nBest Epoch: %d,\nBest performance: %.4f,\nAUC: %.4f\n',tr3.best_epoch,tr3.best_perf,AUC_NN3))
%
%
%%Neural Networks feedforwardnet #2
%
%
RandStream.setGlobalStream (RandStream ('mrg32k3a','Seed', 1234));
trainFcn = 'trainscg';
hiddenLayerSize = 10;
net4 = feedforwardnet(hiddenLayerSize,trainFcn);
net4.performFcn = 'crossentropy';
net4.outputs{numel(hiddenLayerSize)+1}.processParams{2}.ymin = 0;
net4.layers{net4.numLayers}.transferFcn = 'softmax';
net4.plotFcns = {'plotperform','plottrainstate','ploterrhist','plotconfusion','plotroc'};
net4.initFcn='initlay';
net4.layers{1}.initFcn='initnw';
net4.layers{2}.initFcn='initnw';
net4.layers{1}.transferFcn = 'tansig';
net4 = init(net4);
net4.input.processFcns = {'removeconstantrows','mapminmax'};
net4.output.processFcns = {'removeconstantrows','fixunknowns'};
net4.divideFcn = 'divideblock';
net4.divideMode = 'sample';
net4.divideParam.trainRatio = 60/100;
net4.divideParam.valRatio = 20/100;
net4.divideParam.testRatio = 20/100;
net4.trainParam.max_fail = 6;
net4.trainParam.epochs = 1000;
[net4,tr4] = train(net4,x,t);
y4 = sim(net4,x);
PredProb_NN4 = transpose(y4(2,tr4.testInd));
testInd4(tr4.testInd) = true;
[XNN4,YNN4,~,AUC_NN4] = perfcurve(TrainObsNN(testInd4,end), PredProb_NN4, posclass);
display(sprintf('Method: feedforwardnet (#2): \nBest Epoch: %d,\nBest performance: %.4f,\nAUC: %.4f\n',tr4.best_epoch,tr4.best_perf,AUC_NN4))
%
%
if ~isequal(tr.testInd,tr2.testInd) || ~isequal(testInd,testInd2) || ...
~isequal(tr.testInd,tr3.testInd) || ~isequal(tr.testInd,tr4.testInd)
warning('The test sets are not the same!')
end
if isequal(y,y3) && isequal(y2,y4)
display('Random number generation is *not* an issue.')
else
warning('There is a random number generator issue')
end
if isequal(y,y2) || isequal(y3,y4)
display('The two networks are identical')
else
warning('The two netowrks are *not* identical!')
end
Output:
Method: patternnet (#1): Best Epoch: 122, Best performance: 0.2379, AUC: 0.7348
Method: feedforwardnet (#1): Best Epoch: 136, Best performance: 0.2380, AUC: 0.7331
Method: patternnet (#2): Best Epoch: 122, Best performance: 0.2379, AUC: 0.7348
Method: feedforwardnet (#2): Best Epoch: 136, Best performance: 0.2380, AUC: 0.7331
Random number generation is not an issue. Warning: The two netowrks are not identical!
0 Comments
Accepted Answer
Greg Heath
on 21 Oct 2015
Why didn't you just make a simple search on
patternnet feedforwardnet
and find
http://www.mathworks.com/matlabcentral/answers/196799-how-to-simulate-default-patternnet-with-feedforwardnet-in-matlab
Thank you for formally accepting my answer
Greg
0 Comments
More Answers (0)
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!