Neural Network results are same even if I change training/validation ratio?

Dear all,

I am trying to set up a neural network to predict some experimental data.

I created neural network with MATLAB's neural network tool. At first, network results were random and not reproducible. Then I added rng('default') to the beginning of the code to initialize the weights. Then results became reproducible and logical.

Now I realized a different problem in the code. At first training to total ratio was 85/100 and validation to total ratio was 15/100. Even though I play with the ratios, I obtain the same results which is impossible. For example, I change the training 50/100 and val 50/100, still I got the same results.

It seems there is a problem and I wonder the reason. I added to code, could someone explain the reason?

Thank you for your answers.

% Solve an Input-Output Fitting problem with a Neural Network
% Script generated by Neural Fitting app
% Created 09-May-2018 13:53:25
%
% This script assumes these variables are defined:
%
%   Input - input data.
%   output - target data.
clc
clear
rng('default')
x =[data];
x=x';
% 
t=[data2];
t=t';
% Choose a Training Function
% For a list of all training functions type: help nntrain
% 'trainlm' is usually fastest.
% 'trainbr' takes longer but may be better for challenging problems.
% 'trainscg' uses less memory. Suitable in low memory situations.
trainFcn = 'trainbr';  % Levenberg-Marquardt backpropagation.

% Create a Fitting Network
hiddenLayerSize = 4;
net = fitnet(hiddenLayerSize,trainFcn);

% Choose Input and Output Pre/Post-Processing Functions
% For a list of all processing functions type: help nnprocess
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
net.trainParam.epochs = 1000;
% Setup Division of Data for Training, Validation, Testing
% For a list of all data division functions type: help nndivide
net.divideFcn = 'dividerand';  % Divide data randomly
net.divideMode = 'sample';  % Divide up every sample
net.divideParam.trainRatio = 85/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 0/100;

% Choose a Performance Function
% For a list of all performance functions type: help nnperformance
net.performFcn = 'mse';  % Mean Squared Error

% Choose Plot Functions
% For a list of all plot functions type: help nnplot
net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
    'plotregression', 'plotfit'};

% Train the Network
[net,tr] = train(net,x,t);  

% % Test the Network
% y = net(x);
% e = gsubtract(t,y);
% performance = perform(net,t,y)

% Recalculate Training, Validation and Test Performance
% trainTargets = t .* tr.trainMask{1};
% valTargets = t .* tr.valMask{1};
% testTargets = t .* tr.testMask{1};
% trainPerformance = perform(net,trainTargets,y)
% valPerformance = perform(net,valTargets,y)
% testPerformance = perform(net,testTargets,y)

% % View the Network
% view(net)
% 
% % Plots
% Uncomment these lines to enable various plots.
%figure, plotperform(tr)
%figure, plottrainstate(tr)
%figure, ploterrhist(e)
%figure, plotregression(t,y)
%figure, plotfit(net,x,t)

% Deployment
% Change the (false) values to (true) to enable the following code blocks.
% % See the help for each generation function for more information.
% if (false)
%     % Generate MATLAB function for neural network for application
%     % deployment in MATLAB scripts or with MATLAB Compiler and Builder
%     % tools, or simply to examine the calculations your trained neural
%     % network performs.
%     genFunction(net,'myNeuralNetworkFunction');
%     y = myNeuralNetworkFunction(x);
% end
% if (false)
%     % Generate a matrix-only MATLAB function for neural network code
%     % generation with MATLAB Coder tools.
%     genFunction(net,'myNeuralNetworkFunction','MatrixOnly','yes');
%     y = myNeuralNetworkFunction(x);
% end
% if (false)
%     % Generate a Simulink diagram for simulation or deployment with.
%     % Simulink Coder tools.
%     gensim(net);
% end
%

[Greg Heath]

You have not proved the conclusion "I obtained the same results". What are the 6 results

mse(y1-y2), mse(y1-y3) and mse(y2-y3)

for both training and validation?

Greg

Answer 1

[ugur bozuyuk]

Dear Greg,

What I mean was the following: I use "net" function generated from the script to test other data set. I always obtain same result.

[Greg Heath]

I still don't understand. In addition to a more complete description, what EXACTLY does "same result" mean?

Answer 2

[ugur bozuyuk]

First,

I have exactly same test performance values like regression, error histogram etc. Even iteration number is same and I think it should not

Second,

When I use "net" function to predict some experimental data with the created neural network, I will get same numbers. For example, I would like to predict particle size of a nanosystem, I get same particle sizes with different train/val ratios. My data set is proper.