Asked by Shashank Bhatia
on 7 Aug 2018

I am using neural network time series to predict stock price for next week. Normal procedure to approach this problem as explained in nnhelp isn't sufficient. Errors are big and results are absurd. Going through online help, I have seen various answers by Greg which are somewhat helpful but really time consuming. If Greg can write the process for X = phInputs;T = phTargets; example it would be really enlightening. citing steps for 1. Normalize the data (making a time-series stationary), 2. Choosing hidden layer number, 3. Feedback delay number, 4. Data classification (using datablock)

Answer by Greg Heath
on 7 Aug 2018

Accepted Answer

Can Greg post an example for using neural network time series in the whole sense?

Yes he can … and in fact, he has! If the examples are not in ANSWERS, then they must be in COMP.SOFT-SYS.MATLAB!

It may sound strange, but I would first search using the terms

greg ph

If still bewildered, I'm sure you can find similar posts using other MATLAB time-series sample data.

help nndatssets doc nndatasets

**Thank you for formally accepting my answer**

Greg

Shashank Bhatia
on 7 Aug 2018

Shashank Bhatia
on 7 Aug 2018

Greg Heath
on 7 Aug 2018

As you know, I have posted hundreds of NN posts over the years in this and other groups. At this point I have no idea where particular posts are. Also, it is rare that only one of the posts is relevant to a particular question. All I can do is suggest reasonable search words.

The reference nndatasets is where the MATLAB sample data is. It makes no sense for me to spend my time on other data.

Typically, there is more than one relevant reference. I'll check that reference and get back to you.

Sign in to comment.

Answer by Greg Heath
on 9 Aug 2018

Edited by Greg Heath
on 9 Aug 2018

Q1a: Do I need to normalize/standardize the data before feeding to neural network?

A1a: Typically, Yes. One of the following

Normalization : range = [ 0,1 ] Standardization: [mean, variance] = [ 0,1 ]

Q1b: Or does neural network take care for standardization of data

A1b: MATLAB automatically normalizes I prefer to standardize

Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse?

A2: Always plot the data before making any decisions. Then decide what model(s) might be appropriate. You may have to use different models in different ranges.

Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual?

A3a: One hidden layer is always sufficient. Specific knowledge of the data may warrant two. I determine number of hidden nodes by trial and error. A3b. I determine characteristic delays from the auto and crosscorrelation functions

Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?

A4. It depends on which time-series model that you are using. If it is a feedback model it should be obvious that you need to close the loop to predict the future.

A4: Yes. I have posted more than a sufficient number of tutorials in the NEWSGROUP and ANSWERS.

Hope this helps.

*Thank you for formally accepting my answer*

Greg

Sign in to comment.

Answer by Greg Heath
on 9 Aug 2018

Q1a: Do I need to normalize/standardize the data before feeding to neural network?

A1a: Typically, Yes. One of the following

Normalization : range = [ 0,1 ] Standardization: [mean, variance] = [ 0,1 ]

Q1b: Or does neural network take care for standardization of data

A1b: MATLAB automatically normalizes I prefer to standardize

Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse?

A2: Always plot the data before making any decisions. Then decide what model(s) might be appropriate. You may have to use different models in different ranges.

Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual?

A3a: One hidden layer is always sufficient. Specific knowledge of the data may warrant two. I determine number of hidden nodes by trial and error. A3b. I determine characteristic delays from the auto and crosscorrelation functions

Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?

A4: Yes. I have posted more than a sufficient number of tutorials in the NEWSGROUP and ANSWERS.

Hope this helps.

*Thank you for formally accepting my answer*

Greg

Shashank Bhatia
on 10 Aug 2018

Thanks for answering. This has been great guidance.

Coming directly to the further questions:

**FURTHER QUESTION**

Following are the dimensions of NN used. I have no idea how to fix below items: numLayers: 2

numOutputs: 1 numInputDelays: 26 numLayerDelays: 0 numFeedbackDelays: 0 numWeightElements: 20 sampleTime: 1

How these parameters are considered mathematically inside MATLAB?

Searching through NEWSGROUP page, I have found following link to determine ID,FD... but how to decide other factors (layer numbers etc)?

https://groups.google.com/forum/#!searchin/comp.soft-sys.matlab/$20greg$20ph%7Csort:date/comp.soft-sys.matlab/DjDQXsa7wNE/cRhNT1RcBQAJ

**In continuation to our discussion**

Ans 1: Standardization: [mean, variance] = [ 0,1 ] --> Done

Ans 2: Always plot the data before making any decisions. Then decide what model(s) might be appropriate. You may have to use different models in different ranges. --> Understood. In my case I am considering past 1 year data for predicting 1 week ahead stock price closing.

Ans 3:

**Below is my code**:

n=length(data); %10 years data

num=200; % ~1 year data

nextN=5; % to predict next 1 week stock closing

ntrain=n-num;

datatrain=data(ntrain-num:ntrain);

mu = mean(datatrain); %mean closing price of 1 year

sig = std(datatrain); %standard deviation closing price of 1 year

sdatat = (datatrain - mu) / sig; %standarization of data

T=tonndata(sdatat(1:end,1),false,false);

delay=[1:26];

hiddenlayersize=20;

trainFcn='trainlm'

net=narnet(delay,hiddenlayersize,'open',trainFcn);

net.input.processFcns={'removeconstantrows','mapminmax'}

[Xs,Xi,Ai,Ts]=preparets(net,{},{},T);

rng('default')

net=train(net,Xs,Ts,Xi,Ai);

[Ys,Xf,Af]=net(Xs,Xi,Ai);

[netc,Xic,Aic]=closeloop(net,Xf,Af);

nextweek=cell(0,nextN);

Ypred=netc(nextweek,Xic,Aic);

Ypred=transpose(cell2mat(Ypred));

Ypred=Ypred*sig+mu; %Predicted value for 1 week ahead

Ans 4: I have read your posts. Thanks once again.

Sign in to comment.

Answer by Shashank Bhatia
on 8 Aug 2018

Time-series I am working with looks like as per the attachment.

Background: I am using neural network time series to predict stock price (of the attached picture) for next week.Clearly, data is not stationary (time-invariant).

**My questions:**

Q1: Do I need to normalize/standardize the data before feeding to neural network? Or does neural network take care for standardization of data Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse? Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual? Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?

*Thanks in advance.*

Greg Heath
on 9 Aug 2018

I just started reading the above. Will respond as soon possible (am busy on other things now 22:16PM)

Greg

Greg Heath
on 12 Aug 2018

The only thing different about your case is the dominant low order (quadratic, cubic,...?) polynomial trend.

Subtracting that out should yield a decent long term predictor.

Then you can use the remainder for short term predictions.

Greg

Shashank Bhatia
on 13 Aug 2018

With autocorreletion technique I am getting delay value >1000 of my training data number (~2400). For such a high value of delay, required number of nodes is still unclear.

I have used 6 nodes, which is giving me good results (MSE<e-9 for standarized data for open loop) but not always.

**FURTHER QUESTIONS**

1. How to get correct value of Hidden nodes (any approximation would suffice to start with).

2. Right now, I am not training the closed loop or removing delays as you can see in above code, Do I really need to?

Sincerely,

Sign in to comment.

Opportunities for recent engineering grads.

Apply Today
## 3 Comments

## Image Analyst (view profile)

Direct link to this comment:https://www.mathworks.com/matlabcentral/answers/413850-can-greg-post-an-example-for-using-neural-network-time-series-in-the-whole-sense#comment_598987

If Greg can use a

"neural network time series to predict stock price for next week"without results that areaccurateand not "absurd,"he's still here answering your question, then he must be aandwealthy man who must reallyveryMATLAB.love## Greg Heath (view profile)

Direct link to this comment:https://www.mathworks.com/matlabcentral/answers/413850-can-greg-post-an-example-for-using-neural-network-time-series-in-the-whole-sense#comment_598995

I do LOVE MATLAB.

However, if I disclose my prediction method for getting rich, I will eventually be poor and unable to support my family so that they can live far from me.

Hope you understand.

$$$ Great Grandad $$$

## Shashank Bhatia (view profile)

Direct link to this comment:https://www.mathworks.com/matlabcentral/answers/413850-can-greg-post-an-example-for-using-neural-network-time-series-in-the-whole-sense#comment_599030

Had it been possible to predict stock for the next week, no one would be making money. Well stock price isn't only depend on historical data but also on industry trend, PE ratio, news etc etc.

Here my motive is to learn prediction method using NARNET. It seems that "to think that you are investing in real stocks with inputs from neural network" gives one impetus to study deeper.

PS: If People out are making money using simple excel sheet, Quite obviously, Greg can make/is making great grands with his prediction techniques which are calculated risks not gamble.

Thanks Greg for answering and supportive.

Sign in to comment.