Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Forecasting the solar radiation...

Subject: Forecasting the solar radiation...

From: Prodromos

Date: 14 Nov, 2011 11:34:11

Message: 1 of 4

Hi everyone,
it is quite some time now that I try to create a neural network capable of forecasting the future unknown values of solar radiation based on its latest known values. My goal is to predict the total solar radiation of the following day. I am currently using a NAR model from the Neural Network Time Series Tool. The available solar data is 288 samples/day, which I converted into 96 samples/day and 24 samples/day sets for testing purposes.

My best solution refers to a NAR network. At first, the network is created/trained based of the available training data set. Then, the network becomes closed loop (output is directly connected to the input) for multistep prediction. The closed loop network is fed with the "M" latest values of the training set and is used to forecast the next "N" solar radiation values. Then, the predicted values are compared with the real values of the solar radiation, which are completely unknown to the network throughout the whole procedure.

I would also like to add that I have used the NARX model with Air Temperature and Wind Speed as exogenous inputs. Eventually, I abandoned the specific approach due to the inability to make a real forecast, due to the fact that the actual future values of Air Temperature and Wind Speed are also unknown. Thus an iterative multistep prediction could not be implemented for real future values.

Is there any other way/solution to forecast the future values of solar radiation with the minimum possible errors? Please provide me with some code if possible.

My current solution (which does not deliver good results) is the following:

% --------------------------------------------------------------------------------
% Creation and Training of the Network
% --------------------------------------------------------------------------------

% target - feedback time series.
targetSeries = target;

% Create a Nonlinear Autoregressive Network
feedbackDelays = 1:10;
hiddenLayerSize = 30;
net = narnet(feedbackDelays,hiddenLayerSize);

% Feedback Pre/Post-Processing Functions
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};

% Prepare the Data for Training and Simulation
[inputs,inputStates,layerStates,targets] = preparets(net,{},{},targetSeries);

% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand'; % Divide data randomly
net.divideMode = 'time'; % Divide up every value
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Network Training Function
net.trainFcn = 'trainlm';

% Network Performance Function
net.performFcn = 'mse'; % Mean squared error

% Network Plot Functions
net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
  'ploterrcorr', 'plotinerrcorr'};

% Definition of the Training Goal
% net.trainParam.goal = 0;

% Definition of the Network Derivative Function.
net.derivFcn = 'defaultderiv';

% Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);

% Test the Network
outputs = net(inputs,inputStates,layerStates);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)

% Recalculate Training, Validation and Test Performance
trainTargets = gmultiply(targets,tr.trainMask);
valTargets = gmultiply(targets,tr.valMask);
testTargets = gmultiply(targets,tr.testMask);
trainPerformance = perform(net,trainTargets,outputs)
valPerformance = perform(net,valTargets,outputs)
testPerformance = perform(net,testTargets,outputs)

% Closed Loop Network
netc = closeloop(net);


% --------------------------------------------------------------------------------
% Simulation of the Network (Data Forecasting)
% --------------------------------------------------------------------------------

% Deletion of variables (to avoid any exceptions)
clearvars N NetworkInput realValues xc xic aic tc yc prediction RealValues

% Previous Knowledge in Time Steps (Input Delays).
M = 10;

% Forecasting Time Steps.
N = 24;

% Network Input (latest known values)
NetworkInput = target(end-M:end);

% Real Values of Solar Radiation
realValues = test;

% An iteration of the network simulation for N steps. At the end of every
% step, the predicted value is stacked up to the end of the matrix "NetworkInput"
for i = 1:N
    [xc,xic,aic,tc] = preparets(netc,{},{},NetworkInput);
    yc = netc(xc,xic,aic);
    NetworkInput(1,end+1) = yc(1,end);
end

% Forecast vs Real Values Plots
prediction = cell2mat(NetworkInput);
RealValues = cell2mat(realValues(1:length(prediction)));
plot(RealValues,'DisplayName','RealValues','YDataSource','RV');hold all;plot(prediction,'DisplayName','prediction','YDataSource','prediction');hold off;figure(gcf);

Subject: Forecasting the solar radiation...

From: Edo

Date: 11 Oct, 2012 01:52:05

Message: 2 of 4

"Prodromos " <makisffx@gmail.com> wrote in message <j9qubj$reh$1@newscl01ah.mathworks.com>...
> Hi everyone,
> it is quite some time now that I try to create a neural network capable of forecasting the future unknown values of solar radiation based on its latest known values. My goal is to predict the total solar radiation of the following day. I am currently using a NAR model from the Neural Network Time Series Tool. The available solar data is 288 samples/day, which I converted into 96 samples/day and 24 samples/day sets for testing purposes.
>
> My best solution refers to a NAR network. At first, the network is created/trained based of the available training data set. Then, the network becomes closed loop (output is directly connected to the input) for multistep prediction. The closed loop network is fed with the "M" latest values of the training set and is used to forecast the next "N" solar radiation values. Then, the predicted values are compared with the real values of the solar radiation, which are completely unknown to the network throughout the whole procedure.
>
> I would also like to add that I have used the NARX model with Air Temperature and Wind Speed as exogenous inputs. Eventually, I abandoned the specific approach due to the inability to make a real forecast, due to the fact that the actual future values of Air Temperature and Wind Speed are also unknown. Thus an iterative multistep prediction could not be implemented for real future values.
>
> Is there any other way/solution to forecast the future values of solar radiation with the minimum possible errors? Please provide me with some code if possible.
>
> My current solution (which does not deliver good results) is the following:
>
> % --------------------------------------------------------------------------------
> % Creation and Training of the Network
> % --------------------------------------------------------------------------------
>
> % target - feedback time series.
> targetSeries = target;
>
> % Create a Nonlinear Autoregressive Network
> feedbackDelays = 1:10;
> hiddenLayerSize = 30;
> net = narnet(feedbackDelays,hiddenLayerSize);
>
> % Feedback Pre/Post-Processing Functions
> net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
>
> % Prepare the Data for Training and Simulation
> [inputs,inputStates,layerStates,targets] = preparets(net,{},{},targetSeries);
>
> % Setup Division of Data for Training, Validation, Testing
> net.divideFcn = 'dividerand'; % Divide data randomly
> net.divideMode = 'time'; % Divide up every value
> net.divideParam.trainRatio = 70/100;
> net.divideParam.valRatio = 15/100;
> net.divideParam.testRatio = 15/100;
>
> % Network Training Function
> net.trainFcn = 'trainlm';
>
> % Network Performance Function
> net.performFcn = 'mse'; % Mean squared error
>
> % Network Plot Functions
> net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
> 'ploterrcorr', 'plotinerrcorr'};
>
> % Definition of the Training Goal
> % net.trainParam.goal = 0;
>
> % Definition of the Network Derivative Function.
> net.derivFcn = 'defaultderiv';
>
> % Train the Network
> [net,tr] = train(net,inputs,targets,inputStates,layerStates);
>
> % Test the Network
> outputs = net(inputs,inputStates,layerStates);
> errors = gsubtract(targets,outputs);
> performance = perform(net,targets,outputs)
>
> % Recalculate Training, Validation and Test Performance
> trainTargets = gmultiply(targets,tr.trainMask);
> valTargets = gmultiply(targets,tr.valMask);
> testTargets = gmultiply(targets,tr.testMask);
> trainPerformance = perform(net,trainTargets,outputs)
> valPerformance = perform(net,valTargets,outputs)
> testPerformance = perform(net,testTargets,outputs)
>
> % Closed Loop Network
> netc = closeloop(net);
>
>
> % --------------------------------------------------------------------------------
> % Simulation of the Network (Data Forecasting)
> % --------------------------------------------------------------------------------
>
> % Deletion of variables (to avoid any exceptions)
> clearvars N NetworkInput realValues xc xic aic tc yc prediction RealValues
>
> % Previous Knowledge in Time Steps (Input Delays).
> M = 10;
>
> % Forecasting Time Steps.
> N = 24;
>
> % Network Input (latest known values)
> NetworkInput = target(end-M:end);
>
> % Real Values of Solar Radiation
> realValues = test;
>
> % An iteration of the network simulation for N steps. At the end of every
> % step, the predicted value is stacked up to the end of the matrix "NetworkInput"
> for i = 1:N
> [xc,xic,aic,tc] = preparets(netc,{},{},NetworkInput);
> yc = netc(xc,xic,aic);
> NetworkInput(1,end+1) = yc(1,end);
> end
>
> % Forecast vs Real Values Plots
> prediction = cell2mat(NetworkInput);
> RealValues = cell2mat(realValues(1:length(prediction)));
> plot(RealValues,'DisplayName','RealValues','YDataSource','RV');hold all;plot(prediction,'DisplayName','prediction','YDataSource','prediction');hold off;figure(gcf);


Hello,

sorry i can't answer to your questions, but as i am creating a NARX net for a similar problem (predicting Mn(t+1) in lakes knowing Mn(t) and wind, rainfall, T etc at time T), i was wondering while you said that an iterative multispet prediction can not be performed because you din't know the future values of Wind and T; it is true that you dont know them, but at least, you don't give to the net only y(t), but also some more info with x(t)..wouldn't it be better?

Subject: Forecasting the solar radiation...

From: Greg Heath

Date: 12 Oct, 2012 21:06:07

Message: 3 of 4

"Edo" wrote in message <k558o5$kl3$1@newscl01ah.mathworks.com>...
> "Prodromos " <makisffx@gmail.com> wrote in message <j9qubj$reh$1@newscl01ah.mathworks.com>...
> > Hi everyone,
> > it is quite some time now that I try to create a neural network capable of forecasting the future unknown values of solar radiation based on its latest known values. My goal is to predict the total solar radiation of the following day. I am currently using a NAR model from the Neural Network Time Series Tool. The available solar data is 288 samples/day,
> >which I converted into 96 samples/day and 24 samples/day sets for testing purposes.

%How many days of data?

%What about the other 288-96-24 = 120 samples/day?

%Is this data available at a URL?
 
> > My best solution refers to a NAR network. At first, the network is created/trained based of the available training data set. Then, the network becomes closed loop (output is directly connected to the input) for multistep prediction. The closed loop network is fed with the "M" latest values of the training set and is used to forecast the next "N" solar radiation values. Then, the predicted values are compared with the real values of the solar radiation, which are completely unknown to the network throughout the whole procedure.
> >
> > I would also like to add that I have used the NARX model with Air Temperature and Wind Speed as exogenous inputs. Eventually, I abandoned the specific approach due to the inability to make a real forecast, due to the fact that the actual future values of Air Temperature and Wind Speed are also unknown. Thus an iterative multistep prediction could not be implemented for real future values.
> >
> > Is there any other way/solution to forecast the future values of solar radiation with the minimum possible errors? Please provide me with some code if possible.
> >
> > My current solution (which does not deliver good results) is the following:
> >
> > % --------------------------------------------------------------------------------
> > % Creation and Training of the Network
> > % --------------------------------------------------------------------------------
> >
> > % target - feedback time series.
> > targetSeries = target;

whos % Is data double or cell?

[ O N ] =size(targetSeries)

Neq = prod(size(targetSeries)) % Number of equations

Tm = cell2mat(TargetSeries);

%Where are the peaks in the autocorrelation function of Tm?
 
> > % Create a Nonlinear Autoregressive Network
> > feedbackDelays = 1:10;

Consistent with autocorrelation peaks?

> > hiddenLayerSize = 30;

Too large ?

LID = 10, H = 30

Nw = (LID+1)*H+(H+1)*O % = 330+ 31=361 unknown weights

Is Neq >> Nw?

> > net = narnet(feedbackDelays,hiddenLayerSize);

Remove the semicolon so that you can see what default properties
you want to override.

> >
> > % Feedback Pre/Post-Processing Functions
> > net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};

Why needed? Isn't this the default?

> > % Prepare the Data for Training and Simulation
> > [inputs,inputStates,layerStates,targets] = preparets(net,{},{},targetSeries);
> >
> > % Setup Division of Data for Training, Validation, Testing
> > net.divideFcn = 'dividerand'; % Divide data randomly

Why? This ruins the time series data spacing.

Why not 'divideblock' , 'divideint' or even 'divideint' ?

Why waste your time with the next block of commands?
They are all defaults!

-----SNIP


> > % Definition of the Training Goal
> > % net.trainParam.goal = 0;

net.trainParam.goal = 0.01*(Neq-Nw)*var(Tm)/Neq; % Degree-of-freedom adjustment
 
-----SNIP

> > % Train the Network
> > [net,tr] = train(net,inputs,targets,inputStates,layerStates);
> >
> > % Test the Network
> > outputs = net(inputs,inputStates,layerStates);
> > errors = gsubtract(targets,outputs);
> > performance = perform(net,targets,outputs)

% Normalized, DOF adjusted MSE < 0.01 if goal is met

NMSEa = Neq*(performance/var(Tm))/(Neq-Nw)
 
> > % Recalculate Training, Validation and Test Performance
> > trainTargets = gmultiply(targets,tr.trainMask);
> > valTargets = gmultiply(targets,tr.valMask);
> > testTargets = gmultiply(targets,tr.testMask);

Doesn't outpusts have to be masked also?

> > trainPerformance = perform(net,trainTargets,outputs)
> > valPerformance = perform(net,valTargets,outputs)
> > testPerformance = perform(net,testTargets,outputs)

SORRY, cannot help with the closed loop stuff.
 
> > % Closed Loop Network
> > netc = closeloop(net);
> >
> > % --------------------------------------------------------------------------------
> > % Simulation of the Network (Data Forecasting)
> > % --------------------------------------------------------------------------------
> >
> > % Deletion of variables (to avoid any exceptions)
> > clearvars N NetworkInput realValues xc xic aic tc yc prediction RealValues
> >
> > % Previous Knowledge in Time Steps (Input Delays).
> > M = 10;
> >
> > % Forecasting Time Steps.
> > N = 24;
> >
> > % Network Input (latest known values)
> > NetworkInput = target(end-M:end);
> >
> > % Real Values of Solar Radiation
> > realValues = test;
> >
> > % An iteration of the network simulation for N steps. At the end of every
> > % step, the predicted value is stacked up to the end of the matrix "NetworkInput"
> > for i = 1:N
> > [xc,xic,aic,tc] = preparets(netc,{},{},NetworkInput);
> > yc = netc(xc,xic,aic);
> > NetworkInput(1,end+1) = yc(1,end);
> > end
> >
> > % Forecast vs Real Values Plots
> > prediction = cell2mat(NetworkInput);
> > RealValues = cell2mat(realValues(1:length(prediction)));
> > plot(RealValues,'DisplayName','RealValues','YDataSource','RV');hold all;plot(prediction,'DisplayName','prediction','YDataSource','prediction');hold off;figure(gcf);
>
>
> Hello,
>
> sorry i can't answer to your questions, but as i am creating a NARX net for a similar problem (predicting Mn(t+1) in lakes knowing Mn(t) and wind, rainfall, T etc at time T), i was wondering while you said that an iterative multispet prediction can not be performed because you din't know the future values of Wind and T; it is true that you dont know them, but at least, you don't give to the net only y(t), but also some more info with x
>(t)..wouldn't it be better?

Could be. I suggest looking at the peaks of the input-output crosscorrelation function to see if NARX might yield better results than NAR.

Hope this help.

Greg

P.S. I recently learned that PARTIAL auto/crosscorrelations are used by timeseries experts to determine the significant delays. See Wikipedia.

Subject: Forecasting the solar radiation...

From: Gabriel

Date: 7 Oct, 2013 16:10:06

Message: 4 of 4

"Prodromos " <makisffx@gmail.com> wrote in message <j9qubj$reh$1@newscl01ah.mathworks.com>...
> Hi everyone,
> it is quite some time now that I try to create a neural network capable of forecasting the future unknown values of solar radiation based on its latest known values. My goal is to predict the total solar radiation of the following day. I am currently using a NAR model from the Neural Network Time Series Tool. The available solar data is 288 samples/day, which I converted into 96 samples/day and 24 samples/day sets for testing purposes.
>
> My best solution refers to a NAR network. At first, the network is created/trained based of the available training data set. Then, the network becomes closed loop (output is directly connected to the input) for multistep prediction. The closed loop network is fed with the "M" latest values of the training set and is used to forecast the next "N" solar radiation values. Then, the predicted values are compared with the real values of the solar radiation, which are completely unknown to the network throughout the whole procedure.
>
> I would also like to add that I have used the NARX model with Air Temperature and Wind Speed as exogenous inputs. Eventually, I abandoned the specific approach due to the inability to make a real forecast, due to the fact that the actual future values of Air Temperature and Wind Speed are also unknown. Thus an iterative multistep prediction could not be implemented for real future values.
>
> Is there any other way/solution to forecast the future values of solar radiation with the minimum possible errors? Please provide me with some code if possible.
>
> My current solution (which does not deliver good results) is the following:
>
> % --------------------------------------------------------------------------------
> % Creation and Training of the Network
> % --------------------------------------------------------------------------------
>
> % target - feedback time series.
> targetSeries = target;
>
> % Create a Nonlinear Autoregressive Network
> feedbackDelays = 1:10;
> hiddenLayerSize = 30;
> net = narnet(feedbackDelays,hiddenLayerSize);
>
> % Feedback Pre/Post-Processing Functions
> net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
>
> % Prepare the Data for Training and Simulation
> [inputs,inputStates,layerStates,targets] = preparets(net,{},{},targetSeries);
>
> % Setup Division of Data for Training, Validation, Testing
> net.divideFcn = 'dividerand'; % Divide data randomly
> net.divideMode = 'time'; % Divide up every value
> net.divideParam.trainRatio = 70/100;
> net.divideParam.valRatio = 15/100;
> net.divideParam.testRatio = 15/100;
>
> % Network Training Function
> net.trainFcn = 'trainlm';
>
> % Network Performance Function
> net.performFcn = 'mse'; % Mean squared error
>
> % Network Plot Functions
> net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
> 'ploterrcorr', 'plotinerrcorr'};
>
> % Definition of the Training Goal
> % net.trainParam.goal = 0;
>
> % Definition of the Network Derivative Function.
> net.derivFcn = 'defaultderiv';
>
> % Train the Network
> [net,tr] = train(net,inputs,targets,inputStates,layerStates);
>
> % Test the Network
> outputs = net(inputs,inputStates,layerStates);
> errors = gsubtract(targets,outputs);
> performance = perform(net,targets,outputs)
>
> % Recalculate Training, Validation and Test Performance
> trainTargets = gmultiply(targets,tr.trainMask);
> valTargets = gmultiply(targets,tr.valMask);
> testTargets = gmultiply(targets,tr.testMask);
> trainPerformance = perform(net,trainTargets,outputs)
> valPerformance = perform(net,valTargets,outputs)
> testPerformance = perform(net,testTargets,outputs)
>
> % Closed Loop Network
> netc = closeloop(net);
>
>
> % --------------------------------------------------------------------------------
> % Simulation of the Network (Data Forecasting)
> % --------------------------------------------------------------------------------
>
> % Deletion of variables (to avoid any exceptions)
> clearvars N NetworkInput realValues xc xic aic tc yc prediction RealValues
>
> % Previous Knowledge in Time Steps (Input Delays).
> M = 10;
>
> % Forecasting Time Steps.
> N = 24;
>
> % Network Input (latest known values)
> NetworkInput = target(end-M:end);
>
> % Real Values of Solar Radiation
> realValues = test;
>
> % An iteration of the network simulation for N steps. At the end of every
> % step, the predicted value is stacked up to the end of the matrix "NetworkInput"
> for i = 1:N
> [xc,xic,aic,tc] = preparets(netc,{},{},NetworkInput);
> yc = netc(xc,xic,aic);
> NetworkInput(1,end+1) = yc(1,end);
> end
>
> % Forecast vs Real Values Plots
> prediction = cell2mat(NetworkInput);
> RealValues = cell2mat(realValues(1:length(prediction)));
> plot(RealValues,'DisplayName','RealValues','YDataSource','RV');hold all;plot(prediction,'DisplayName','prediction','YDataSource','prediction');hold off;figure(gcf);



Hi,
It's been 2 years since you posted this question. where you able to find the code to forecast with NN toolbox? I'm kinda doing something very similar to what you did (also working with solar information) and I need to forecast that data. Could you please send me the code?
My data has about 44000 steps (basically 1min timestep for 1month) and I want to forecast about 15steps for short-term solar forecasting..
Thank you so much ;)

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us