Path: news.mathworks.com!not-for-mail
From: "Subodh Paudel" <subodhpaudel@gmail.com>
Newsgroups: comp.soft-sys.matlab
Subject: Threads in  Neural Network in Train/Validation/Test
Date: Thu, 21 Feb 2013 20:07:08 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 31
Message-ID: <kg5upc$pgj$1@newscl01ah.mathworks.com>
Reply-To: "Subodh Paudel" <subodhpaudel@gmail.com>
NNTP-Posting-Host: www-04-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: newscl01ah.mathworks.com 1361477228 26131 172.30.248.35 (21 Feb 2013 20:07:08 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Thu, 21 Feb 2013 20:07:08 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 3983627
Xref: news.mathworks.com comp.soft-sys.matlab:789563

Dear All,
I am using MATLAB R2009a. I have a different answer for train/validation/test from two different method: 

1) I Use: 
[net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);

 to train the network,   and simulate the different train/validation/test result as: 

normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
normValidateOutput=sim(net,valV.P,[],[],valV.T);
normTestOutput=sim(net,testV.P,[],[],testV.T);

and then i obtained MSE for training validation and test as:

MSETrain=tr.perf(end);
MSEValidate=tr.vperf(end);
MSETest=tr.tperf(end);

And from them finally R2 square value as: 

R2Train=1-NMSETrain
R2Validate=1-NMSEValidate
R2Test=1-NMSETest

And the result i obtained directly from MSEtrain1=mse(normTrainOutput - tn(:XX)), that starts from training interval period i defined. And so on validation and test. Why these two values MSETrain1 and MSETrain differ?

2) I have R2 Train = 0.7738, R2 Validate = 0.7934 and R2 Test = 0.7926. And from the linear regression plot i obtain R train = 0.89584, R validate = 0.81805 and R Test = 0.92432. Does it mean the R2 value of neural network is worst than linear regression model? OR the result i obtained during training = 0.89584 from regression is quite good.

3) Every times i simulate my network, my R2 values sometimes good and sometimes even worst  -ve. How to make it constant, if i assume i get 27 epochs,  hidden neurons  =18 the best R2 value?

Thank You.