Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
What activation function should I choose?

Subject: What activation function should I choose?

From: Arga Ridhalla

Date: 31 Jan, 2013 14:48:08

Message: 1 of 2

Hi everybody!
I use Neural Network Toolbox for a time series prediction (using NARX with 'traingdm' training function), and now I'm still confuse to choose an activation function for input and output layer. I have normalized all variables (including target variable) so now the values range between 0-1. What activation function should I choose for input and output layer? Is it tansig, logsig, or purelin? And what's the difference between them?

Thanks for the answers and sorry for my bad English.

Subject: What activation function should I choose?

From: Greg Heath

Date: 3 Feb, 2013 21:13:06

Message: 2 of 2

"Arga Ridhalla" <arga.ridhalla@yahoo.co.id> wrote in message <kee078$ftj$1@newscl01ah.mathworks.com>...
> Hi everybody!
> I use Neural Network Toolbox for a time series prediction (using NARX with 'traingdm' training function), and now I'm still confuse to choose an activation function for input and output layer. I have normalized all variables (including target variable) so now the values range between 0-1. What activation function should I choose for input and output layer? Is it tansig, logsig, or purelin? And what's the difference between them?

Did you ever think of plotting them over x= -10:10 ?

The choice depends on the expected bounds of the output data (see the graphs).
>
> Thanks for the answers and sorry for my bad English.

Always start with documentation examples and function defaults. If the example defaults do not work with your data, then change them one at a time. See the documentation and example at

help narxnet
...
 narxnet(inputDelays,feedbackDelays,hiddenSizes,feedbackMode,trainFcn)
...
Defaults are used if narxnet is called with fewer arguments.
   The default arguments are (1:2,1:2,10,'open','trainlm').
...
 net = narxnet(1:2,1:2,10);

This final command is missing:

MSE = perform(net,Ts,Y)

which is best understood in normalized form

MSE00 = mean(var(cell2mat(Ts),1))
NMSE = MSE/MSE00
=============================================
Also see the documentation and different example at

doc narxnet
...
narxnet(inputDelays,hiddenSizes,trainFcn) <== BUG !!!
...
feedbackDelays Row vector of increasing 0 or positive delays (default = 1:2) <= BUG !!!

BUG ==> ZERO FEEDBACK DELAYS ARE NOT ALLOWED
================================================

Hope this helps.

Greg

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us