Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Neural Network Building Energy Forecast

Subject: Neural Network Building Energy Forecast

From: Subodh Paudel

Date: 14 Feb, 2013 14:48:14

Message: 1 of 5

Hai All,
I want to forecast the building energy demand: input parameters: outside temperature and solar radiation, and output parameter: energy consumption

I am choosing 2 layered network. I have a 1826 samples. My R2 value is in negative and i am not finding the solution of it? Please, could you help me.

The data size consists all inputs and targets in column vector. I have used 70% samples for the generalization and 30% for prediction.

The souce code is given:

data=xlsread('z:\testdata.xlsx','Sheet1')
%Size of input and Target Layers I:Input Layer, O:Output Layer, dN=size of
%the data
[dN I]=size(data);
dNt=round(0.7*dN); %70% of Data Used for the Training and Generalization

input_sample= data((1:dNt),1:2)';
target_sample=data((1:dNt),3)';
validation_inputsample=data(((dNt+1):dN),1:2)';
validation_targetsample=data(((dNt+1):dN),3)';

%DataProcessing of Training Sample (normalization with mean 0.1 and 0.9)
[pn,ps]=mapstd(input_sample,0.1,0.9);
[tn,ts]=mapstd(target_sample,0.1,0.9);

%Apply Normalization to the Validation Input Sample
an=mapstd('apply',validation_inputsample,ps);

%Iterative Optimization wrt to RMSE to determine Hidden Neurons:hidden_node,
%Learning Rate:lr_rate and Momentum:mc_rate
sbmin=99999999;
hidden_node=0;
mc_rate=0;
lr_rate=0;
mm=1;
%MLP (2-1-1 Network)with tangent hyperbolic transfer function in the hidden layer and
%Linear Activation Function in the Output Layer
%
%Initial Hidden Neuron Size Varies from 5 to 10
for i=1:1:40;
    net=newff(minmax(pn),[i 1],{'tansig','purelin'},'trainlm')
      net.performFcn='mse';
      net.performParam=[];
      net.trainParam.goal=0.01*var(tn);
      net.trainParam.epochs=1000;
      net.trainParam.show=30;
     net.trainParam.mu_max=1e10;
     net.trainParam.min_grad=0.000001;
     net.trainParam.max_fail=20;
    for k=1:1:4 %Learning Rate Varies from 0.25 to 1
        lr=0.25+(k-1)*.25;
        net.trainParam.lr=lr;
        for m=1:1:4 %Momentum Varies from 0.25 to 1
            mc=0.25+(m-1)*0.25;
            net.trainParam.m=mc;
            net=init(net);%Assume 0 weight
            net=train(net,pn,tn);
            yn=sim(net,pn);
            y=mapstd('reverse',yn,ts);
            error=y-target_sample;
            RMSE(i,k,m)=sqrt(mse(error));%RMSE Value Check for the Minimum Root Mean Square
            if RMSE(i,k,m)<sbmin
                sbmin=RMSE(i,k,m);
                hidden_node=i;
                mc_rate=mc;
                lr_rate=lr;
                 IW_initial=net.IW{1,1};
               LW_initial=net.IW{2,1};
                hidden_bias_initial=net.b{1};
                output_bias_initial=net.b{2};
            end
        end
    end
end
parameters =[hidden_node mc_rate lr_rate]
net=newff(minmax(pn),[hidden_node 1],{'tansig','purelin'},'trainlm')
 net.performFcn='mse';
 net.performParam=[];
net.trainParam.goal=0.01*var(tn);
net.trainParam.epochs=1000;
net.trainParam.show=30;
 net.trainParam.mu_max=1e10;
net.trainParam.max_fail=20;
net.trainParam.min_grad=0.000001;
net.trainParam.lr=lr_rate;
net.trainParam.m=mc_rate;

MSE_initial=mean(var(target_sample)); %Approximate MSE for a constant output model
net.IW{1,1}=IW_initial;
 net.IW{2,1}=LW_initial;
 net.b{1}=hidden_bias_initial;
 net.b{2}=output_bias_initial;

net=train(net,pn,tn);
yn=sim(net,an);
y=mapstd('reverse',yn,ts);
%Error Calculation
error=y-validation_targetsample;
percentage_error=abs(error)./y*100; %Percentage of Error
RMSE=sqrt(mse(error)); %Root Mean Square Error
NMSE= mse(error)/MSE_initial; % Normalized Mean Square Error
R2=1-NMSE; %R^2 Statistics for the Regression Analysis
MAE=mean(abs(error)); %Mean Absolute Error
MAPE = mean(percentage_error); % Mean Absolute Percentage Error
%Analysis of Predicted and Target Values
performance_error=[RMSE R2]
pp=[validation_targetsample y];
figure(1)
plot(y,'g')
hold
plot(validation_targetsample,'r')
title('comparision between actual targets and predictions')

Subject: Neural Network Building Energy Forecast

From: Greg Heath

Date: 15 Feb, 2013 11:40:07

Message: 2 of 5

"Subodh Paudel" <subodhpaudel@gmail.com> wrote in message
<kfitfe$8hi$1@newscl01ah.mathworks.com>...
> Hai All,
> I want to forecast the building energy demand: input parameters:
outside temperature and solar radiation, and output parameter: energy
consumption
>
> I am choosing 2 layered network. I have a 1826 samples. My R2
value is in negative and i am not finding the solution of it? Please,
could you help me.
>
> The data size consists all inputs and targets in column vector.

[ I N ] = size( input ) % [ 2 1826 ]
[ O N ] = size( target ) % [ 1 1826 ]

>I have used 70% samples for the generalization and 30% for prediction.

Terminology:

total = design + test
design = train + validation

> The souce code is given:
>
> data=xlsread('z:\testdata.xlsx','Sheet1')
> %Size of input and Target Layers I:Input Layer, O:Output Layer,

Official Terminology: 2 layers of Hidden and Output neurons. Input
elements are fan-in units and NOT neurons. The unmodifed term
'layer' does not apply to inputs. The typical MLP is a 2-layer FFNN.
                     
I don't like this at all. Therefore I only refer to (3) layers of nodes (NOT
neurons!). Consequently, since the typical MLP always has a layer of
input nodes and a layer of output nodes, I refer to the typical topology
as a MLP or FFNN with one hidden layer (of nodes or neurons).

dN=size of
> %the data
> [dN I]=size(data);

Please remove the "d" it is unnecessary and confusing.

> dNt=round(0.7*dN); %70% of Data Used for the Training and
Generalization

Not standard terminology or proceedure
Design = Training and Validation
Use Validation set iteratively to determine training parameters and
choose best of multiple designs.
Need a nondesign subset (Test) to predict generalization error.
If results are unsatisfactory, need to repeat with a new trn/val/tst
data split (i.e., Test set is only used ONCE!)

> input_sample= data((1:dNt),1:2)';
> target_sample=data((1:dNt),3)';
> validation_inputsample=data(((dNt+1):dN),1:2)';
> validation_targetsample=data(((dNt+1):dN),3)';
>
> %DataProcessing of Training Sample (normalization with
mean 0.1 and 0.9)
> [pn,ps]=mapstd(input_sample,0.1,0.9);
> [tn,ts]=mapstd(target_sample,0.1,0.9);

No! There is absolutely no good reason to do that!
Standarization means zero-mean/unit-variance.

> %Apply Normalization to the Validation Input Sample
> an=mapstd('apply',validation_inputsample,ps);
>
> %Iterative Optimization wrt to RMSE to determine Hidden
Neurons:hidden_node,

Use dimensionless NMSE = MSE/MSE00 (MSE00 =
mean(var(target',1)) ) instead of scale dependent RMSE.
Then coefficient of determination (i.e., R^2) is given by
R2 =1-NMSE.

> %Learning Rate:lr_rate and Momentum:mc_rate
> sbmin=99999999;
> hidden_node=0;
> mc_rate=0;
> lr_rate=0;
> mm=1;

??
Always begin with the MATLAB defaults.

> %MLP (2-1-1 Network)with tangent hyperbolic transfer function in
the hidden layer and
> %Linear Activation Function in the Output Layer

TERMINOLOGY: 2-H-1 node topology

> %Initial Hidden Neuron Size Varies from 5 to 10

Ntrneq = Ntrn*O % No. of training equations
Nw = (I+1)*H+(H+1)*O % No. of unknown weights

To mitigate noise and measurement error when not
using regularization(msereg) or validation stopping,
require Ntrneq >= Nw but desire Ntrneq >> Nw.
This yields an upperbound for H

Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)) % 319

%Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
%Hmax ~ round(Hub/8) = 40

> for i=1:1:40;
    
    How about starting with a coarse search 1:5:40
    
> net=newff(minmax(pn),[i 1],{'tansig','purelin'},'trainlm')
> net.performFcn='mse';
> net.performParam=[];
> net.trainParam.goal=0.01*var(tn);
> net.trainParam.epochs=1000;
> net.trainParam.show=30;
> net.trainParam.mu_max=1e10;
> net.trainParam.min_grad=0.000001;

Too complicated. To start, use as many defaults as possible
For example

net=newff(minmax(pn),[i 1]);
net.trainParam.goal=0.01*var(tn);
net.trainParam.show=30;

> net.trainParam.max_fail=20;

Too high for val stopping. Just delete and accept the default of 6.

> for k=1:1:4 %Learning Rate Varies from 0.25 to 1
> lr=0.25+(k-1)*.25;
> net.trainParam.lr=lr;
> for m=1:1:4 %Momentum Varies from 0.25 to 1
> mc=0.25+(m-1)*0.25;
> net.trainParam.m=mc;
> net=init(net);%Assume 0 weight
> net=train(net,pn,tn);
> yn=sim(net,pn);
> y=mapstd('reverse',yn,ts);
> error=y-target_sample;
> RMSE(i,k,m)=sqrt(mse(error));%RMSE Value Check for
the Minimum Root Mean Square
> if RMSE(i,k,m)<sbmin
> sbmin=RMSE(i,k,m);
> hidden_node=i;
> mc_rate=mc;
> lr_rate=lr;
> IW_initial=net.IW{1,1};
> LW_initial=net.IW{2,1};
> hidden_bias_initial=net.b{1};
> output_bias_initial=net.b{2};
> end
> end
> end
> end

??? Life is already hard enough, why not accept most defaults
and just search over H and initial weights?

NOTE: minmax version of newff automatically configures the
initial weights.

I'll stop here. Basically, start simple and gradually make
improvements. Your current code will probably take hours and
not give you any insight.

Hope this helps.

Greg

> parameters =[hidden_node mc_rate lr_rate]
> net=newff(minmax(pn),[hidden_node 1],{'tansig','purelin'},'trainlm')
> net.performFcn='mse';
> net.performParam=[];
> net.trainParam.goal=0.01*var(tn);
> net.trainParam.epochs=1000;
> net.trainParam.show=30;
> net.trainParam.mu_max=1e10;
> net.trainParam.max_fail=20;
> net.trainParam.min_grad=0.000001;
> net.trainParam.lr=lr_rate;
> net.trainParam.m=mc_rate;
>
> MSE_initial=mean(var(target_sample)); %Approximate MSE for
a constant output model
> net.IW{1,1}=IW_initial;
> net.IW{2,1}=LW_initial;
> net.b{1}=hidden_bias_initial;
> net.b{2}=output_bias_initial;
>
> net=train(net,pn,tn);
> yn=sim(net,an);
> y=mapstd('reverse',yn,ts);
> %Error Calculation
> error=y-validation_targetsample;
> percentage_error=abs(error)./y*100; %Percentage of Error
> RMSE=sqrt(mse(error)); %Root Mean Square Error
> NMSE= mse(error)/MSE_initial; % Normalized Mean Square Error
> R2=1-NMSE; %R^2 Statistics for the Regression Analysis
> MAE=mean(abs(error)); %Mean Absolute Error
> MAPE = mean(percentage_error); % Mean Absolute Percentage
Error
> %Analysis of Predicted and Target Values
> performance_error=[RMSE R2]
> pp=[validation_targetsample y];
> figure(1)
> plot(y,'g')
> hold
> plot(validation_targetsample,'r')
> title('comparision between actual targets and predictions'

Subject: Neural Network Building Energy Forecast

From: subodhpaudel@gmail.com

Date: 15 Feb, 2013 13:31:52

Message: 3 of 5

On Friday, February 15, 2013 12:40:07 PM UTC+1, Greg Heath wrote:
> "Subodh Paudel" <subodhpaudel@gmail.com> wrote in message
>
> <kfitfe$8hi$1@newscl01ah.mathworks.com>...
>
> > Hai All,
>
> > I want to forecast the building energy demand: input parameters:
>
> outside temperature and solar radiation, and output parameter: energy
>
> consumption
>
> >
>
> > I am choosing 2 layered network. I have a 1826 samples. My R2
>
> value is in negative and i am not finding the solution of it? Please,
>
> could you help me.
>
> >
>
> > The data size consists all inputs and targets in column vector.
>
>
>
> [ I N ] = size( input ) % [ 2 1826 ]
>
> [ O N ] = size( target ) % [ 1 1826 ]
>
>
>
> >I have used 70% samples for the generalization and 30% for prediction.
>
>
>
> Terminology:
>
>
>
> total = design + test
>
> design = train + validation
>
>
>
> > The souce code is given:
>
> >
>
> > data=xlsread('z:\testdata.xlsx','Sheet1')
>
> > %Size of input and Target Layers I:Input Layer, O:Output Layer,
>
>
>
> Official Terminology: 2 layers of Hidden and Output neurons. Input
>
> elements are fan-in units and NOT neurons. The unmodifed term
>
> 'layer' does not apply to inputs. The typical MLP is a 2-layer FFNN.
>
>
>
> I don't like this at all. Therefore I only refer to (3) layers of nodes (NOT
>
> neurons!). Consequently, since the typical MLP always has a layer of
>
> input nodes and a layer of output nodes, I refer to the typical topology
>
> as a MLP or FFNN with one hidden layer (of nodes or neurons).
>
>
>
> dN=size of
>
> > %the data
>
> > [dN I]=size(data);
>
>
>
> Please remove the "d" it is unnecessary and confusing.
>
>
>
> > dNt=round(0.7*dN); %70% of Data Used for the Training and
>
> Generalization
>
>
>
> Not standard terminology or proceedure
>
> Design = Training and Validation
>
> Use Validation set iteratively to determine training parameters and
>
> choose best of multiple designs.
>
> Need a nondesign subset (Test) to predict generalization error.
>
> If results are unsatisfactory, need to repeat with a new trn/val/tst
>
> data split (i.e., Test set is only used ONCE!)
>
>
>
> > input_sample= data((1:dNt),1:2)';
>
> > target_sample=data((1:dNt),3)';
>
> > validation_inputsample=data(((dNt+1):dN),1:2)';
>
> > validation_targetsample=data(((dNt+1):dN),3)';
>
> >
>
> > %DataProcessing of Training Sample (normalization with
>
> mean 0.1 and 0.9)
>
> > [pn,ps]=mapstd(input_sample,0.1,0.9);
>
> > [tn,ts]=mapstd(target_sample,0.1,0.9);
>
>
>
> No! There is absolutely no good reason to do that!
>
> Standarization means zero-mean/unit-variance.
>
>
>
> > %Apply Normalization to the Validation Input Sample
>
> > an=mapstd('apply',validation_inputsample,ps);
>
> >
>
> > %Iterative Optimization wrt to RMSE to determine Hidden
>
> Neurons:hidden_node,
>
>
>
> Use dimensionless NMSE = MSE/MSE00 (MSE00 =
>
> mean(var(target',1)) ) instead of scale dependent RMSE.
>
> Then coefficient of determination (i.e., R^2) is given by
>
> R2 =1-NMSE.
>
>
>
> > %Learning Rate:lr_rate and Momentum:mc_rate
>
> > sbmin=99999999;
>
> > hidden_node=0;
>
> > mc_rate=0;
>
> > lr_rate=0;
>
> > mm=1;
>
>
>
> ??
>
> Always begin with the MATLAB defaults.
>
>
>
> > %MLP (2-1-1 Network)with tangent hyperbolic transfer function in
>
> the hidden layer and
>
> > %Linear Activation Function in the Output Layer
>
>
>
> TERMINOLOGY: 2-H-1 node topology
>
>
>
> > %Initial Hidden Neuron Size Varies from 5 to 10
>
>
>
> Ntrneq = Ntrn*O % No. of training equations
>
> Nw = (I+1)*H+(H+1)*O % No. of unknown weights
>
>
>
> To mitigate noise and measurement error when not
>
> using regularization(msereg) or validation stopping,
>
> require Ntrneq >= Nw but desire Ntrneq >> Nw.
>
> This yields an upperbound for H
>
>
>
> Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)) % 319
>
>
>
> %Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
>
> %Hmax ~ round(Hub/8) = 40
>
>
>
> > for i=1:1:40;
>
>
>
> How about starting with a coarse search 1:5:40
>
>
>
> > net=newff(minmax(pn),[i 1],{'tansig','purelin'},'trainlm')
>
> > net.performFcn='mse';
>
> > net.performParam=[];
>
> > net.trainParam.goal=0.01*var(tn);
>
> > net.trainParam.epochs=1000;
>
> > net.trainParam.show=30;
>
> > net.trainParam.mu_max=1e10;
>
> > net.trainParam.min_grad=0.000001;
>
>
>
> Too complicated. To start, use as many defaults as possible
>
> For example
>
>
>
> net=newff(minmax(pn),[i 1]);
>
> net.trainParam.goal=0.01*var(tn);
>
> net.trainParam.show=30;
>
>
>
> > net.trainParam.max_fail=20;
>
>
>
> Too high for val stopping. Just delete and accept the default of 6.
>
>
>
> > for k=1:1:4 %Learning Rate Varies from 0.25 to 1
>
> > lr=0.25+(k-1)*.25;
>
> > net.trainParam.lr=lr;
>
> > for m=1:1:4 %Momentum Varies from 0.25 to 1
>
> > mc=0.25+(m-1)*0.25;
>
> > net.trainParam.m=mc;
>
> > net=init(net);%Assume 0 weight
>
> > net=train(net,pn,tn);
>
> > yn=sim(net,pn);
>
> > y=mapstd('reverse',yn,ts);
>
> > error=y-target_sample;
>
> > RMSE(i,k,m)=sqrt(mse(error));%RMSE Value Check for
>
> the Minimum Root Mean Square
>
> > if RMSE(i,k,m)<sbmin
>
> > sbmin=RMSE(i,k,m);
>
> > hidden_node=i;
>
> > mc_rate=mc;
>
> > lr_rate=lr;
>
> > IW_initial=net.IW{1,1};
>
> > LW_initial=net.IW{2,1};
>
> > hidden_bias_initial=net.b{1};
>
> > output_bias_initial=net.b{2};
>
> > end
>
> > end
>
> > end
>
> > end
>
>
>
> ??? Life is already hard enough, why not accept most defaults
>
> and just search over H and initial weights?
>
>
>
> NOTE: minmax version of newff automatically configures the
>
> initial weights.
>
>
>
> I'll stop here. Basically, start simple and gradually make
>
> improvements. Your current code will probably take hours and
>
> not give you any insight.
>
>
>
> Hope this helps.
>
>
>
> Greg
>
>
>
> > parameters =[hidden_node mc_rate lr_rate]
>
> > net=newff(minmax(pn),[hidden_node 1],{'tansig','purelin'},'trainlm')
>
> > net.performFcn='mse';
>
> > net.performParam=[];
>
> > net.trainParam.goal=0.01*var(tn);
>
> > net.trainParam.epochs=1000;
>
> > net.trainParam.show=30;
>
> > net.trainParam.mu_max=1e10;
>
> > net.trainParam.max_fail=20;
>
> > net.trainParam.min_grad=0.000001;
>
> > net.trainParam.lr=lr_rate;
>
> > net.trainParam.m=mc_rate;
>
> >
>
> > MSE_initial=mean(var(target_sample)); %Approximate MSE for
>
> a constant output model
>
> > net.IW{1,1}=IW_initial;
>
> > net.IW{2,1}=LW_initial;
>
> > net.b{1}=hidden_bias_initial;
>
> > net.b{2}=output_bias_initial;
>
> >
>
> > net=train(net,pn,tn);
>
> > yn=sim(net,an);
>
> > y=mapstd('reverse',yn,ts);
>
> > %Error Calculation
>
> > error=y-validation_targetsample;
>
> > percentage_error=abs(error)./y*100; %Percentage of Error
>
> > RMSE=sqrt(mse(error)); %Root Mean Square Error
>
> > NMSE= mse(error)/MSE_initial; % Normalized Mean Square Error
>
> > R2=1-NMSE; %R^2 Statistics for the Regression Analysis
>
> > MAE=mean(abs(error)); %Mean Absolute Error
>
> > MAPE = mean(percentage_error); % Mean Absolute Percentage
>
> Error
>
> > %Analysis of Predicted and Target Values
>
> > performance_error=[RMSE R2]
>
> > pp=[validation_targetsample y];
>
> > figure(1)
>
> > plot(y,'g')
>
> > hold
>
> > plot(validation_targetsample,'r')
>
> > title('comparision between actual targets and predictions'


Thank you very much Greg. I will correct my code from your suggestions and came to you.

Subject: Neural Network Building Energy Forecast

From: Subodh Paudel

Date: 18 Feb, 2013 21:02:06

Message: 4 of 5

Hai Greg,
Thank You for your previous suggestion. Accordingly, i changed my training structure and still the building energy forecast is not good.
I have some queries, further,:
1) All the time i run, i have the R-value i.e. in the regression plot in the neural network changing? I could not figure it out...is the weight changed all the time...
2) Regarding MSE00 i.e. initial condition, MSE00=mean(var(target)), it has a large value in the range of exp.. does it should have a normalize value?
3) How can i read the R2 value of the test, validation and training value although matlab describes the value for all the cases:

Here is My Rest Code:

data=xlsread('C:\Users\Dell\Desktop\testdata.xlsx','Sheet1');
[dN dI]=size(data);

%Input and Target Data
P= data(1:dN,1:2)';
T=data(1:dN,3)';

%Input and Hidden Layer
[I N]=size(P); %[2 1826]
[O N]=size(T); %[1 1826]
% MSE00=mean(var(T));

%Normalization of Input and Output Data
[pn,ps]=mapstd(P);
[tn,ts]=mapstd(T);

H=8; %(Iterative Optimization)
Ntrneq=N*O; %1826 No of Training Equations
Nw = (I+1)*H+(H+1)*O; % No. of unknown weights
%This yields an upperbound for H
Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)); % 319
%Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
%Hmax ~ round(Hub/8) = 40

%MLP (2-H-1)
net=newff(minmax(pn),[H O]);

%Divide Samples into Training, Validation and Test
valPercent=0.015;
testPercent=0.015;
[trainV, valV, testV]=dividevec(pn,tn,valPercent, testPercent);

%Parameters Setting for Learning and Training
% net.trainParam.goal=0.01*MSE00;
net.trainParam.epochs=1000;

%Train the Network
[net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);

%Simulate the Network
normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
normValidateOutput=sim(net,valV.P,[],[],valV.T);
normTestOutput=sim(net,testV.P,[],[],testV.T);

%Reverse Normalize Outputs
trainOutput=mapstd('reverse',normTrainOutput,ts);
validateOutput=mapstd('reverse',normValidateOutput,ts);
testOutput=mapstd('reverse',normTestOutput,ts);

%Plot the Predicted Values to the Actual Values
[s sTr]=size(trainOutput);
[s sVa]=size(validateOutput);
[s sTe]=size(testOutput);

figure(1)
plot(testOutput,'b')
hold on
plot(T((sTr+sVa+1):N),'k')
title('comparision between Training Actual and Predictions')

Again, Thank you for reading and feedback.
























































subodhpaudel@gmail.com wrote in message <b3df16d2-4414-4ce1-aadd-528cb7f0a7d0@googlegroups.com>...
> On Friday, February 15, 2013 12:40:07 PM UTC+1, Greg Heath wrote:
> > "Subodh Paudel" <subodhpaudel@gmail.com> wrote in message
> >
> > <kfitfe$8hi$1@newscl01ah.mathworks.com>...
> >
> > > Hai All,
> >
> > > I want to forecast the building energy demand: input parameters:
> >
> > outside temperature and solar radiation, and output parameter: energy
> >
> > consumption
> >
> > >
> >
> > > I am choosing 2 layered network. I have a 1826 samples. My R2
> >
> > value is in negative and i am not finding the solution of it? Please,
> >
> > could you help me.
> >
> > >
> >
> > > The data size consists all inputs and targets in column vector.
> >
> >
> >
> > [ I N ] = size( input ) % [ 2 1826 ]
> >
> > [ O N ] = size( target ) % [ 1 1826 ]
> >
> >
> >
> > >I have used 70% samples for the generalization and 30% for prediction.
> >
> >
> >
> > Terminology:
> >
> >
> >
> > total = design + test
> >
> > design = train + validation
> >
> >
> >
> > > The souce code is given:
> >
> > >
> >
> > > data=xlsread('z:\testdata.xlsx','Sheet1')
> >
> > > %Size of input and Target Layers I:Input Layer, O:Output Layer,
> >
> >
> >
> > Official Terminology: 2 layers of Hidden and Output neurons. Input
> >
> > elements are fan-in units and NOT neurons. The unmodifed term
> >
> > 'layer' does not apply to inputs. The typical MLP is a 2-layer FFNN.
> >
> >
> >
> > I don't like this at all. Therefore I only refer to (3) layers of nodes (NOT
> >
> > neurons!). Consequently, since the typical MLP always has a layer of
> >
> > input nodes and a layer of output nodes, I refer to the typical topology
> >
> > as a MLP or FFNN with one hidden layer (of nodes or neurons).
> >
> >
> >
> > dN=size of
> >
> > > %the data
> >
> > > [dN I]=size(data);
> >
> >
> >
> > Please remove the "d" it is unnecessary and confusing.
> >
> >
> >
> > > dNt=round(0.7*dN); %70% of Data Used for the Training and
> >
> > Generalization
> >
> >
> >
> > Not standard terminology or proceedure
> >
> > Design = Training and Validation
> >
> > Use Validation set iteratively to determine training parameters and
> >
> > choose best of multiple designs.
> >
> > Need a nondesign subset (Test) to predict generalization error.
> >
> > If results are unsatisfactory, need to repeat with a new trn/val/tst
> >
> > data split (i.e., Test set is only used ONCE!)
> >
> >
> >
> > > input_sample= data((1:dNt),1:2)';
> >
> > > target_sample=data((1:dNt),3)';
> >
> > > validation_inputsample=data(((dNt+1):dN),1:2)';
> >
> > > validation_targetsample=data(((dNt+1):dN),3)';
> >
> > >
> >
> > > %DataProcessing of Training Sample (normalization with
> >
> > mean 0.1 and 0.9)
> >
> > > [pn,ps]=mapstd(input_sample,0.1,0.9);
> >
> > > [tn,ts]=mapstd(target_sample,0.1,0.9);
> >
> >
> >
> > No! There is absolutely no good reason to do that!
> >
> > Standarization means zero-mean/unit-variance.
> >
> >
> >
> > > %Apply Normalization to the Validation Input Sample
> >
> > > an=mapstd('apply',validation_inputsample,ps);
> >
> > >
> >
> > > %Iterative Optimization wrt to RMSE to determine Hidden
> >
> > Neurons:hidden_node,
> >
> >
> >
> > Use dimensionless NMSE = MSE/MSE00 (MSE00 =
> >
> > mean(var(target',1)) ) instead of scale dependent RMSE.
> >
> > Then coefficient of determination (i.e., R^2) is given by
> >
> > R2 =1-NMSE.
> >
> >
> >
> > > %Learning Rate:lr_rate and Momentum:mc_rate
> >
> > > sbmin=99999999;
> >
> > > hidden_node=0;
> >
> > > mc_rate=0;
> >
> > > lr_rate=0;
> >
> > > mm=1;
> >
> >
> >
> > ??
> >
> > Always begin with the MATLAB defaults.
> >
> >
> >
> > > %MLP (2-1-1 Network)with tangent hyperbolic transfer function in
> >
> > the hidden layer and
> >
> > > %Linear Activation Function in the Output Layer
> >
> >
> >
> > TERMINOLOGY: 2-H-1 node topology
> >
> >
> >
> > > %Initial Hidden Neuron Size Varies from 5 to 10
> >
> >
> >
> > Ntrneq = Ntrn*O % No. of training equations
> >
> > Nw = (I+1)*H+(H+1)*O % No. of unknown weights
> >
> >
> >
> > To mitigate noise and measurement error when not
> >
> > using regularization(msereg) or validation stopping,
> >
> > require Ntrneq >= Nw but desire Ntrneq >> Nw.
> >
> > This yields an upperbound for H
> >
> >
> >
> > Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)) % 319
> >
> >
> >
> > %Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
> >
> > %Hmax ~ round(Hub/8) = 40
> >
> >
> >
> > > for i=1:1:40;
> >
> >
> >
> > How about starting with a coarse search 1:5:40
> >
> >
> >
> > > net=newff(minmax(pn),[i 1],{'tansig','purelin'},'trainlm')
> >
> > > net.performFcn='mse';
> >
> > > net.performParam=[];
> >
> > > net.trainParam.goal=0.01*var(tn);
> >
> > > net.trainParam.epochs=1000;
> >
> > > net.trainParam.show=30;
> >
> > > net.trainParam.mu_max=1e10;
> >
> > > net.trainParam.min_grad=0.000001;
> >
> >
> >
> > Too complicated. To start, use as many defaults as possible
> >
> > For example
> >
> >
> >
> > net=newff(minmax(pn),[i 1]);
> >
> > net.trainParam.goal=0.01*var(tn);
> >
> > net.trainParam.show=30;
> >
> >
> >
> > > net.trainParam.max_fail=20;
> >
> >
> >
> > Too high for val stopping. Just delete and accept the default of 6.
> >
> >
> >
> > > for k=1:1:4 %Learning Rate Varies from 0.25 to 1
> >
> > > lr=0.25+(k-1)*.25;
> >
> > > net.trainParam.lr=lr;
> >
> > > for m=1:1:4 %Momentum Varies from 0.25 to 1
> >
> > > mc=0.25+(m-1)*0.25;
> >
> > > net.trainParam.m=mc;
> >
> > > net=init(net);%Assume 0 weight
> >
> > > net=train(net,pn,tn);
> >
> > > yn=sim(net,pn);
> >
> > > y=mapstd('reverse',yn,ts);
> >
> > > error=y-target_sample;
> >
> > > RMSE(i,k,m)=sqrt(mse(error));%RMSE Value Check for
> >
> > the Minimum Root Mean Square
> >
> > > if RMSE(i,k,m)<sbmin
> >
> > > sbmin=RMSE(i,k,m);
> >
> > > hidden_node=i;
> >
> > > mc_rate=mc;
> >
> > > lr_rate=lr;
> >
> > > IW_initial=net.IW{1,1};
> >
> > > LW_initial=net.IW{2,1};
> >
> > > hidden_bias_initial=net.b{1};
> >
> > > output_bias_initial=net.b{2};
> >
> > > end
> >
> > > end
> >
> > > end
> >
> > > end
> >
> >
> >
> > ??? Life is already hard enough, why not accept most defaults
> >
> > and just search over H and initial weights?
> >
> >
> >
> > NOTE: minmax version of newff automatically configures the
> >
> > initial weights.
> >
> >
> >
> > I'll stop here. Basically, start simple and gradually make
> >
> > improvements. Your current code will probably take hours and
> >
> > not give you any insight.
> >
> >
> >
> > Hope this helps.
> >
> >
> >
> > Greg
> >
> >
> >
> > > parameters =[hidden_node mc_rate lr_rate]
> >
> > > net=newff(minmax(pn),[hidden_node 1],{'tansig','purelin'},'trainlm')
> >
> > > net.performFcn='mse';
> >
> > > net.performParam=[];
> >
> > > net.trainParam.goal=0.01*var(tn);
> >
> > > net.trainParam.epochs=1000;
> >
> > > net.trainParam.show=30;
> >
> > > net.trainParam.mu_max=1e10;
> >
> > > net.trainParam.max_fail=20;
> >
> > > net.trainParam.min_grad=0.000001;
> >
> > > net.trainParam.lr=lr_rate;
> >
> > > net.trainParam.m=mc_rate;
> >
> > >
> >
> > > MSE_initial=mean(var(target_sample)); %Approximate MSE for
> >
> > a constant output model
> >
> > > net.IW{1,1}=IW_initial;
> >
> > > net.IW{2,1}=LW_initial;
> >
> > > net.b{1}=hidden_bias_initial;
> >
> > > net.b{2}=output_bias_initial;
> >
> > >
> >
> > > net=train(net,pn,tn);
> >
> > > yn=sim(net,an);
> >
> > > y=mapstd('reverse',yn,ts);
> >
> > > %Error Calculation
> >
> > > error=y-validation_targetsample;
> >
> > > percentage_error=abs(error)./y*100; %Percentage of Error
> >
> > > RMSE=sqrt(mse(error)); %Root Mean Square Error
> >
> > > NMSE= mse(error)/MSE_initial; % Normalized Mean Square Error
> >
> > > R2=1-NMSE; %R^2 Statistics for the Regression Analysis
> >
> > > MAE=mean(abs(error)); %Mean Absolute Error
> >
> > > MAPE = mean(percentage_error); % Mean Absolute Percentage
> >
> > Error
> >
> > > %Analysis of Predicted and Target Values
> >
> > > performance_error=[RMSE R2]
> >
> > > pp=[validation_targetsample y];
> >
> > > figure(1)
> >
> > > plot(y,'g')
> >
> > > hold
> >
> > > plot(validation_targetsample,'r')
> >
> > > title('comparision between actual targets and predictions'
>
>
> Thank you very much Greg. I will correct my code from your suggestions and came to you.

Subject: Neural Network Building Energy Forecast

From: Subodh Paudel

Date: 19 Feb, 2013 22:40:11

Message: 5 of 5

"Subodh Paudel" <subodhpaudel@gmail.com> wrote in message <kfu4se$6s6$1@newscl01ah.mathworks.com>...
> Hai Greg,
> Thank You for your previous suggestion. Accordingly, i changed my training structure and still the building energy forecast is not good.
> I have some queries, further,:
> 1) All the time i run, i have the R-value i.e. in the regression plot in the neural network changing? I could not figure it out...is the weight changed all the time...
> 2) Regarding MSE00 i.e. initial condition, MSE00=mean(var(target)), it has a large value in the range of exp.. does it should have a normalize value?
> 3) How can i read the R2 value of the test, validation and training value although matlab describes the value for all the cases:
>
> Here is My Rest Code:
>
> data=xlsread('C:\Users\Dell\Desktop\testdata.xlsx','Sheet1');
> [dN dI]=size(data);
>
> %Input and Target Data
> P= data(1:dN,1:2)';
> T=data(1:dN,3)';
>
> %Input and Hidden Layer
> [I N]=size(P); %[2 1826]
> [O N]=size(T); %[1 1826]
> % MSE00=mean(var(T));
>
> %Normalization of Input and Output Data
> [pn,ps]=mapstd(P);
> [tn,ts]=mapstd(T);
>
> H=8; %(Iterative Optimization)
> Ntrneq=N*O; %1826 No of Training Equations
> Nw = (I+1)*H+(H+1)*O; % No. of unknown weights
> %This yields an upperbound for H
> Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)); % 319
> %Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
> %Hmax ~ round(Hub/8) = 40
>
> %MLP (2-H-1)
> net=newff(minmax(pn),[H O]);
>
> %Divide Samples into Training, Validation and Test
> valPercent=0.015;
> testPercent=0.015;
> [trainV, valV, testV]=dividevec(pn,tn,valPercent, testPercent);
>
> %Parameters Setting for Learning and Training
> % net.trainParam.goal=0.01*MSE00;
> net.trainParam.epochs=1000;
>
> %Train the Network
> [net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);
>
> %Simulate the Network
> normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
> normValidateOutput=sim(net,valV.P,[],[],valV.T);
> normTestOutput=sim(net,testV.P,[],[],testV.T);
>
> %Reverse Normalize Outputs
> trainOutput=mapstd('reverse',normTrainOutput,ts);
> validateOutput=mapstd('reverse',normValidateOutput,ts);
> testOutput=mapstd('reverse',normTestOutput,ts);
>
> %Plot the Predicted Values to the Actual Values
> [s sTr]=size(trainOutput);
> [s sVa]=size(validateOutput);
> [s sTe]=size(testOutput);
>
> figure(1)
> plot(testOutput,'b')
> hold on
> plot(T((sTr+sVa+1):N),'k')
> title('comparision between Training Actual and Predictions')
>
> Again, Thank you for reading and feedback.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> subodhpaudel@gmail.com wrote in message <b3df16d2-4414-4ce1-aadd-528cb7f0a7d0@googlegroups.com>...
> > On Friday, February 15, 2013 12:40:07 PM UTC+1, Greg Heath wrote:
> > > "Subodh Paudel" <subodhpaudel@gmail.com> wrote in message
> > >
> > > <kfitfe$8hi$1@newscl01ah.mathworks.com>...
> > >
> > > > Hai All,
> > >
> > > > I want to forecast the building energy demand: input parameters:
> > >
> > > outside temperature and solar radiation, and output parameter: energy
> > >
> > > consumption
> > >
> > > >
> > >
> > > > I am choosing 2 layered network. I have a 1826 samples. My R2
> > >
> > > value is in negative and i am not finding the solution of it? Please,
> > >
> > > could you help me.
> > >
> > > >
> > >
> > > > The data size consists all inputs and targets in column vector.
> > >
> > >
> > >
> > > [ I N ] = size( input ) % [ 2 1826 ]
> > >
> > > [ O N ] = size( target ) % [ 1 1826 ]
> > >
> > >
> > >
> > > >I have used 70% samples for the generalization and 30% for prediction.
> > >
> > >
> > >
> > > Terminology:
> > >
> > >
> > >
> > > total = design + test
> > >
> > > design = train + validation
> > >
> > >
> > >
> > > > The souce code is given:
> > >
> > > >
> > >
> > > > data=xlsread('z:\testdata.xlsx','Sheet1')
> > >
> > > > %Size of input and Target Layers I:Input Layer, O:Output Layer,
> > >
> > >
> > >
> > > Official Terminology: 2 layers of Hidden and Output neurons. Input
> > >
> > > elements are fan-in units and NOT neurons. The unmodifed term
> > >
> > > 'layer' does not apply to inputs. The typical MLP is a 2-layer FFNN.
> > >
> > >
> > >
> > > I don't like this at all. Therefore I only refer to (3) layers of nodes (NOT
> > >
> > > neurons!). Consequently, since the typical MLP always has a layer of
> > >
> > > input nodes and a layer of output nodes, I refer to the typical topology
> > >
> > > as a MLP or FFNN with one hidden layer (of nodes or neurons).
> > >
> > >
> > >
> > > dN=size of
> > >
> > > > %the data
> > >
> > > > [dN I]=size(data);
> > >
> > >
> > >
> > > Please remove the "d" it is unnecessary and confusing.
> > >
> > >
> > >
> > > > dNt=round(0.7*dN); %70% of Data Used for the Training and
> > >
> > > Generalization
> > >
> > >
> > >
> > > Not standard terminology or proceedure
> > >
> > > Design = Training and Validation
> > >
> > > Use Validation set iteratively to determine training parameters and
> > >
> > > choose best of multiple designs.
> > >
> > > Need a nondesign subset (Test) to predict generalization error.
> > >
> > > If results are unsatisfactory, need to repeat with a new trn/val/tst
> > >
> > > data split (i.e., Test set is only used ONCE!)
> > >
> > >
> > >
> > > > input_sample= data((1:dNt),1:2)';
> > >
> > > > target_sample=data((1:dNt),3)';
> > >
> > > > validation_inputsample=data(((dNt+1):dN),1:2)';
> > >
> > > > validation_targetsample=data(((dNt+1):dN),3)';
> > >
> > > >
> > >
> > > > %DataProcessing of Training Sample (normalization with
> > >
> > > mean 0.1 and 0.9)
> > >
> > > > [pn,ps]=mapstd(input_sample,0.1,0.9);
> > >
> > > > [tn,ts]=mapstd(target_sample,0.1,0.9);
> > >
> > >
> > >
> > > No! There is absolutely no good reason to do that!
> > >
> > > Standarization means zero-mean/unit-variance.
> > >
> > >
> > >
> > > > %Apply Normalization to the Validation Input Sample
> > >
> > > > an=mapstd('apply',validation_inputsample,ps);
> > >
> > > >
> > >
> > > > %Iterative Optimization wrt to RMSE to determine Hidden
> > >
> > > Neurons:hidden_node,
> > >
> > >
> > >
> > > Use dimensionless NMSE = MSE/MSE00 (MSE00 =
> > >
> > > mean(var(target',1)) ) instead of scale dependent RMSE.
> > >
> > > Then coefficient of determination (i.e., R^2) is given by
> > >
> > > R2 =1-NMSE.
> > >
> > >
> > >
> > > > %Learning Rate:lr_rate and Momentum:mc_rate
> > >
> > > > sbmin=99999999;
> > >
> > > > hidden_node=0;
> > >
> > > > mc_rate=0;
> > >
> > > > lr_rate=0;
> > >
> > > > mm=1;
> > >
> > >
> > >
> > > ??
> > >
> > > Always begin with the MATLAB defaults.
> > >
> > >
> > >
> > > > %MLP (2-1-1 Network)with tangent hyperbolic transfer function in
> > >
> > > the hidden layer and
> > >
> > > > %Linear Activation Function in the Output Layer
> > >
> > >
> > >
> > > TERMINOLOGY: 2-H-1 node topology
> > >
> > >
> > >
> > > > %Initial Hidden Neuron Size Varies from 5 to 10
> > >
> > >
> > >
> > > Ntrneq = Ntrn*O % No. of training equations
> > >
> > > Nw = (I+1)*H+(H+1)*O % No. of unknown weights
> > >
> > >
> > >
> > > To mitigate noise and measurement error when not
> > >
> > > using regularization(msereg) or validation stopping,
> > >
> > > require Ntrneq >= Nw but desire Ntrneq >> Nw.
> > >
> > > This yields an upperbound for H
> > >
> > >
> > >
> > > Hub = -1 + ceil( (Ntrneq-O)/(I+O+1)) % 319
> > >
> > >
> > >
> > > %Hub = -1 + ceil( ( 0.7*1826-1)/(2+1+1) ) = 319
> > >
> > > %Hmax ~ round(Hub/8) = 40
> > >
> > >
> > >
> > > > for i=1:1:40;
> > >
> > >
> > >
> > > How about starting with a coarse search 1:5:40
> > >
> > >
> > >
> > > > net=newff(minmax(pn),[i 1],{'tansig','purelin'},'trainlm')
> > >
> > > > net.performFcn='mse';
> > >
> > > > net.performParam=[];
> > >
> > > > net.trainParam.goal=0.01*var(tn);
> > >
> > > > net.trainParam.epochs=1000;
> > >
> > > > net.trainParam.show=30;
> > >
> > > > net.trainParam.mu_max=1e10;
> > >
> > > > net.trainParam.min_grad=0.000001;
> > >
> > >
> > >
> > > Too complicated. To start, use as many defaults as possible
> > >
> > > For example
> > >
> > >
> > >
> > > net=newff(minmax(pn),[i 1]);
> > >
> > > net.trainParam.goal=0.01*var(tn);
> > >
> > > net.trainParam.show=30;
> > >
> > >
> > >
> > > > net.trainParam.max_fail=20;
> > >
> > >
> > >
> > > Too high for val stopping. Just delete and accept the default of 6.
> > >
> > >
> > >
> > > > for k=1:1:4 %Learning Rate Varies from 0.25 to 1
> > >
> > > > lr=0.25+(k-1)*.25;
> > >
> > > > net.trainParam.lr=lr;
> > >
> > > > for m=1:1:4 %Momentum Varies from 0.25 to 1
> > >
> > > > mc=0.25+(m-1)*0.25;
> > >
> > > > net.trainParam.m=mc;
> > >
> > > > net=init(net);%Assume 0 weight
> > >
> > > > net=train(net,pn,tn);
> > >
> > > > yn=sim(net,pn);
> > >
> > > > y=mapstd('reverse',yn,ts);
> > >
> > > > error=y-target_sample;
> > >
> > > > RMSE(i,k,m)=sqrt(mse(error));%RMSE Value Check for
> > >
> > > the Minimum Root Mean Square
> > >
> > > > if RMSE(i,k,m)<sbmin
> > >
> > > > sbmin=RMSE(i,k,m);
> > >
> > > > hidden_node=i;
> > >
> > > > mc_rate=mc;
> > >
> > > > lr_rate=lr;
> > >
> > > > IW_initial=net.IW{1,1};
> > >
> > > > LW_initial=net.IW{2,1};
> > >
> > > > hidden_bias_initial=net.b{1};
> > >
> > > > output_bias_initial=net.b{2};
> > >
> > > > end
> > >
> > > > end
> > >
> > > > end
> > >
> > > > end
> > >
> > >
> > >
> > > ??? Life is already hard enough, why not accept most defaults
> > >
> > > and just search over H and initial weights?
> > >
> > >
> > >
> > > NOTE: minmax version of newff automatically configures the
> > >
> > > initial weights.
> > >
> > >
> > >
> > > I'll stop here. Basically, start simple and gradually make
> > >
> > > improvements. Your current code will probably take hours and
> > >
> > > not give you any insight.
> > >
> > >
> > >
> > > Hope this helps.
> > >
> > >
> > >
> > > Greg
> > >
> > >
> > >
> > > > parameters =[hidden_node mc_rate lr_rate]
> > >
> > > > net=newff(minmax(pn),[hidden_node 1],{'tansig','purelin'},'trainlm')
> > >
> > > > net.performFcn='mse';
> > >
> > > > net.performParam=[];
> > >
> > > > net.trainParam.goal=0.01*var(tn);
> > >
> > > > net.trainParam.epochs=1000;
> > >
> > > > net.trainParam.show=30;
> > >
> > > > net.trainParam.mu_max=1e10;
> > >
> > > > net.trainParam.max_fail=20;
> > >
> > > > net.trainParam.min_grad=0.000001;
> > >
> > > > net.trainParam.lr=lr_rate;
> > >
> > > > net.trainParam.m=mc_rate;
> > >
> > > >
> > >
> > > > MSE_initial=mean(var(target_sample)); %Approximate MSE for
> > >
> > > a constant output model
> > >
> > > > net.IW{1,1}=IW_initial;
> > >
> > > > net.IW{2,1}=LW_initial;
> > >
> > > > net.b{1}=hidden_bias_initial;
> > >
> > > > net.b{2}=output_bias_initial;
> > >
> > > >
> > >
> > > > net=train(net,pn,tn);
> > >
> > > > yn=sim(net,an);
> > >
> > > > y=mapstd('reverse',yn,ts);
> > >
> > > > %Error Calculation
> > >
> > > > error=y-validation_targetsample;
> > >
> > > > percentage_error=abs(error)./y*100; %Percentage of Error
> > >
> > > > RMSE=sqrt(mse(error)); %Root Mean Square Error
> > >
> > > > NMSE= mse(error)/MSE_initial; % Normalized Mean Square Error
> > >
> > > > R2=1-NMSE; %R^2 Statistics for the Regression Analysis
> > >
> > > > MAE=mean(abs(error)); %Mean Absolute Error
> > >
> > > > MAPE = mean(percentage_error); % Mean Absolute Percentage
> > >
> > > Error
> > >
> > > > %Analysis of Predicted and Target Values
> > >
> > > > performance_error=[RMSE R2]
> > >
> > > > pp=[validation_targetsample y];
> > >
> > > > figure(1)
> > >
> > > > plot(y,'g')
> > >
> > > > hold
> > >
> > > > plot(validation_targetsample,'r')
> > >
> > > > title('comparision between actual targets and predictions'
> >
> >
> > Thank you very much Greg. I will correct my code from your suggestions and came to you.




Dear Sir Greg,
Thank you very much for your previous suggestion and the your precious time given to me. I have forget to address some of the queries also. Accordingly, I changed my training structure. Here is the code i changed, but still, i found many problems. the prediction pattern follows, however, R2 value for training is 0.62 but R2 value is negative for the training and validation phase. Please, could you address my some questionaries below:

Here is the code:
Here is My Rest Code:

%Input and Hidden Layer
[I N]=size(P); %[2 1826]
[O N]=size(T); %[1 1826]

MSE00=mean(var(T)); % (1) Is this correct ?

%Normalization of Input and Output Data
[pn,ps]=mapstd(P); %Already Transposed
[tn,ts]=mapstd(T);

H=18; %(Iterative Optimization)
%MLP (2-H-1)

net=newff(minmax(pn),[H O]);

%Divide Samples into Training, Validation and Test
valPercent=0.015;
testPercent=0.015;
[trainV, valV, testV]=dividevec(pn,tn,valPercent, testPercent);

%Parameters Setting for Learning and Training
    net.trainParam.goal=0.01*MSE00;
    net.trainParam.epochs=1000;

%Train the Network
[net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);

%Simulate the Network
normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
normValidateOutput=sim(net,valV.P,[],[],valV.T);
normTestOutput=sim(net,testV.P,[],[],testV.T);

%Reverse Normalize Outputs
trainOutput=mapstd('reverse',normTrainOutput,ts);
validateOutput=mapstd('reverse',normValidateOutput,ts);
testOutput=mapstd('reverse',normTestOutput,ts);

% (2) Is this a correct way to calculate MSE and R2 values for training, testing and validation. I am Using Matlab 2009a??? Because MSE00 is the global for all, how to compute MSE00 for train, validatation and test??

[s sTr]=size(trainOutput);
[s sVa]=size(validateOutput);
[s sTe]=size(testOutput);

errorTrain=(T(1:sTr))-trainOutput;
MSETrain=mse(errorTrain);
R2Train=1-MSETrain/MSE00; % (3) Is this should be divided by MSE00 ?? %0.624

errorValidate=(T((sTr+1):(sTr+sVa)))-validateOutput;
MSEValidate=mse(errorValidate);
R2Validate=1-MSEValidate/MSE00; %-3.245

errorTest=(T((sTr+sVa+1):N))-testOutput;
MSETest=mse(errorTest);
R2Test=1-MSETest/MSE00; %-1.245

%(4) What is the differences between R2 train/val/test this correlation value compare to the neural network shown shown R value for train/val/test? Because in the neural network, i have this value 0.89/0.86/0.84 for train/val/test? Is the Neural network shown is Naive Linear Model?

%(5) Although i achieved even this value, my R value is always changing when i simulate? Is this due to change of weight. Assume, i get satisfactory value, how can i generalize further to predict other new values??

figure(1)
plot(testOutput,'b')
hold on
plot(T((sTr+sVa+1):N),'k')
title('comparision between Training Actual and Predictions')

Again, Thank you for reading and feedback.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us