Can Greg post an example for using neural network time series in the whole sense?

4 views (last 30 days)
I am using neural network time series to predict stock price for next week. Normal procedure to approach this problem as explained in nnhelp isn't sufficient. Errors are big and results are absurd. Going through online help, I have seen various answers by Greg which are somewhat helpful but really time consuming. If Greg can write the process for X = phInputs;T = phTargets; example it would be really enlightening. citing steps for 1. Normalize the data (making a time-series stationary), 2. Choosing hidden layer number, 3. Feedback delay number, 4. Data classification (using datablock)
  4 Comments
Shashank Bhatia
Shashank Bhatia on 13 Aug 2018
Had it been possible to predict stock for the next week, no one would be making money. Well stock price isn't only depend on historical data but also on industry trend, PE ratio, news etc etc.
Here my motive is to learn prediction method using NARNET. It seems that "to think that you are investing in real stocks with inputs from neural network" gives one impetus to study deeper.
PS: If People out are making money using simple excel sheet, Quite obviously, Greg can make/is making great grands with his prediction techniques which are calculated risks not gamble.
Thanks Greg for answering and supportive.
KAE
KAE on 23 Oct 2018
Edited: KAE on 23 Oct 2018
This is the funniest thing I have seen on Answers. Greg, we will double the fee you get for answering questions if you give a failsafe stock predictor.

Sign in to comment.

Accepted Answer

Greg Heath
Greg Heath on 7 Aug 2018
Can Greg post an example for using neural network time series in the whole sense?
Yes he can … and in fact, he has! If the examples are not in ANSWERS, then they must be in COMP.SOFT-SYS.MATLAB!
It may sound strange, but I would first search using the terms
greg ph
If still bewildered, I'm sure you can find similar posts using other MATLAB time-series sample data.
help nndatssets
doc nndatasets
Thank you for formally accepting my answer
Greg
  4 Comments
Greg Heath
Greg Heath on 7 Aug 2018
As you know, I have posted hundreds of NN posts over the years in this and other groups. At this point I have no idea where particular posts are. Also, it is rare that only one of the posts is relevant to a particular question. All I can do is suggest reasonable search words.
The reference nndatasets is where the MATLAB sample data is. It makes no sense for me to spend my time on other data.
Typically, there is more than one relevant reference. I'll check that reference and get back to you.

Sign in to comment.

More Answers (3)

Greg Heath
Greg Heath on 9 Aug 2018
Edited: Greg Heath on 9 Aug 2018
Q1a: Do I need to normalize/standardize the data before feeding to neural network?
A1a: Typically, Yes. One of the following
Normalization : range = [ 0,1 ]
Standardization: [mean, variance] = [ 0,1 ]
Q1b: Or does neural network take care for standardization of data
A1b: MATLAB automatically normalizes
I prefer to standardize
Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse?
A2: Always plot the data before making any decisions.
Then decide what model(s) might be appropriate.
You may have to use different models in different
ranges.
Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual?
A3a: One hidden layer is always sufficient.
Specific knowledge of the data may warrant
two. I determine number of hidden nodes by
trial and error.
A3b. I determine characteristic delays from the
auto and crosscorrelation functions
Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?
A4. It depends on which time-series model that you are using.
If it is a feedback model it should be obvious that you need to
close the loop to predict the future.
A4: Yes. I have posted more than a sufficient number
of tutorials in the NEWSGROUP and ANSWERS.
Hope this helps.
*Thank you for formally accepting my answer*
Greg

Greg Heath
Greg Heath on 9 Aug 2018
Q1a: Do I need to normalize/standardize the data before feeding to neural network?
A1a: Typically, Yes. One of the following
Normalization : range = [ 0,1 ]
Standardization: [mean, variance] = [ 0,1 ]
Q1b: Or does neural network take care for standardization of data
A1b: MATLAB automatically normalizes
I prefer to standardize
Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse?
A2: Always plot the data before making any decisions.
Then decide what model(s) might be appropriate.
You may have to use different models in different
ranges.
Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual?
A3a: One hidden layer is always sufficient. Specific
knowledge of the data may warrant two. I determine
number of hidden nodes by trial and error.
A3b. I determine characteristic delays from the auto
and crosscorrelation functions
Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?
A4: Yes. I have posted more than a sufficient number
of tutorials in the NEWSGROUP and ANSWERS.
Hope this helps.
*Thank you for formally accepting my answer*
Greg
  1 Comment
Shashank Bhatia
Shashank Bhatia on 10 Aug 2018
Edited: Shashank Bhatia on 12 Aug 2018
Thanks for answering. This has been great guidance.
Coming directly to the further questions:
FURTHER QUESTION
Following are the dimensions of NN used. I have no idea how to fix below items: numLayers: 2
numOutputs: 1
numInputDelays: 26
numLayerDelays: 0
numFeedbackDelays: 0
numWeightElements: 20
sampleTime: 1
How these parameters are considered mathematically inside MATLAB?
Searching through NEWSGROUP page, I have found following link to determine ID,FD... but how to decide other factors (layer numbers etc)?
https://groups.google.com/forum/#!searchin/comp.soft-sys.matlab/$20greg$20ph%7Csort:date/comp.soft-sys.matlab/DjDQXsa7wNE/cRhNT1RcBQAJ
In continuation to our discussion
Ans 1: Standardization: [mean, variance] = [ 0,1 ] --> Done
Ans 2: Always plot the data before making any decisions. Then decide what model(s) might be appropriate. You may have to use different models in different ranges. --> Understood. In my case I am considering past 1 year data for predicting 1 week ahead stock price closing.
Ans 3:
Below is my code:
n=length(data); %10 years data
num=200; % ~1 year data
nextN=5; % to predict next 1 week stock closing
ntrain=n-num;
datatrain=data(ntrain-num:ntrain);
mu = mean(datatrain); %mean closing price of 1 year
sig = std(datatrain); %standard deviation closing price of 1 year
sdatat = (datatrain - mu) / sig; %standarization of data
T=tonndata(sdatat(1:end,1),false,false);
delay=[1:26];
hiddenlayersize=20;
trainFcn='trainlm'
net=narnet(delay,hiddenlayersize,'open',trainFcn);
net.input.processFcns={'removeconstantrows','mapminmax'}
[Xs,Xi,Ai,Ts]=preparets(net,{},{},T);
rng('default')
net=train(net,Xs,Ts,Xi,Ai);
[Ys,Xf,Af]=net(Xs,Xi,Ai);
[netc,Xic,Aic]=closeloop(net,Xf,Af);
nextweek=cell(0,nextN);
Ypred=netc(nextweek,Xic,Aic);
Ypred=transpose(cell2mat(Ypred));
Ypred=Ypred*sig+mu; %Predicted value for 1 week ahead
Ans 4: I have read your posts. Thanks once again.

Sign in to comment.


Shashank Bhatia
Shashank Bhatia on 8 Aug 2018
Time-series I am working with looks like as per the attachment.
Background: I am using neural network time series to predict stock price (of the attached picture) for next week.Clearly, data is not stationary (time-invariant).
My questions:
Q1: Do I need to normalize/standardize the data before feeding to neural network? Or does neural network take care for standardization of data Q2: How can I decide range of the data to be used? 5 yrs or 10 years? Is the process of doing so manual observing model mse? Q3: How can I decide number of hidden layers, FD (feedback delays)? Is the process manual? Q4: After making the network with sufficiently accurate mse, do I need to convert the net into closed-loop (netc) for next week prediction?
Thanks in advance.
  3 Comments
Greg Heath
Greg Heath on 12 Aug 2018
The only thing different about your case is the dominant low order (quadratic, cubic,...?) polynomial trend.
Subtracting that out should yield a decent long term predictor.
Then you can use the remainder for short term predictions.
Greg
Shashank Bhatia
Shashank Bhatia on 13 Aug 2018
Edited: Shashank Bhatia on 13 Aug 2018
With autocorreletion technique I am getting delay value >1000 of my training data number (~2400). For such a high value of delay, required number of nodes is still unclear.
I have used 6 nodes, which is giving me good results (MSE<e-9 for standarized data for open loop) but not always.
FURTHER QUESTIONS
1. How to get correct value of Hidden nodes (any approximation would suffice to start with).
2. Right now, I am not training the closed loop or removing delays as you can see in above code, Do I really need to?
Sincerely,

Sign in to comment.

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!