Path: news.mathworks.com!not-for-mail
From: "Greg Heath" <heath@alumni.brown.edu>
Newsgroups: comp.soft-sys.matlab
Subject: Re: What activation function should I choose?
Date: Sun, 3 Feb 2013 21:13:06 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 44
Message-ID: <kemjt2$15b$1@newscl01ah.mathworks.com>
References: <kee078$ftj$1@newscl01ah.mathworks.com>
Reply-To: "Greg Heath" <heath@alumni.brown.edu>
NNTP-Posting-Host: www-03-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: newscl01ah.mathworks.com 1359925986 1195 172.30.248.48 (3 Feb 2013 21:13:06 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sun, 3 Feb 2013 21:13:06 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 2929937
Xref: news.mathworks.com comp.soft-sys.matlab:788129

"Arga Ridhalla" <arga.ridhalla@yahoo.co.id> wrote in message <kee078$ftj$1@newscl01ah.mathworks.com>...
> Hi everybody!
> I use Neural Network Toolbox for a time series prediction (using NARX with 'traingdm' training function), and now I'm still confuse to choose an activation function for input and output layer. I have normalized all variables (including target variable) so now the values range between 0-1. What activation function should I choose for input and output layer? Is it tansig, logsig, or purelin? And what's the difference between them?

Did you ever think of plotting them over x= -10:10 ?

The choice depends on the expected bounds of the output data (see the graphs).
> 
> Thanks for the answers and sorry for my bad English. 

Always start with documentation examples and function defaults. If the example defaults do not work with your data, then change them one at a time. See the documentation and example at

help narxnet
...
 narxnet(inputDelays,feedbackDelays,hiddenSizes,feedbackMode,trainFcn)
...
Defaults are used if narxnet is called with fewer arguments.
   The default arguments are (1:2,1:2,10,'open','trainlm').
...
 net = narxnet(1:2,1:2,10);

This final command is missing:

MSE = perform(net,Ts,Y)

which is best understood in normalized form

MSE00 = mean(var(cell2mat(Ts),1))
NMSE = MSE/MSE00
=============================================
Also see the documentation and different example at

doc narxnet
...
narxnet(inputDelays,hiddenSizes,trainFcn)    <==   BUG !!! 
...
feedbackDelays Row vector of increasing 0 or positive delays (default = 1:2) <=  BUG !!!

BUG ==> ZERO FEEDBACK DELAYS ARE NOT ALLOWED 
================================================

Hope this helps.

Greg