Neural Network Output :Scaling the output range.

13 views (last 30 days)
Hi,
The output layer of my neural network (3 layered) is using sigmoid as activation which outputs only in range [0-1]. However, if I want to train it for outputs that are beyond [0-1], say in thousands, what should I do?
For example if I want to train
input ----> output
0 0 ------> 0
0 1 ------> 1000
1000 1 ----> 1
1 1 -------> 0
My program works for AND, OR, XOR etc. As input output are all in binary.
There were some suggestion to use,
Activation:
-----------
y = lambda*(abs(x)*1/(1+exp(-1*(x))))
Derivative of activation:
-------------------------
lambda*(abs(y)*y*(1-y))
This did not converge for the mentioned training pattern. Are there any suggestion please?

Accepted Answer

Greg Heath
Greg Heath on 31 Jan 2012
Hello Greg,
Thanks again for answering the question. For my case, no rigid bound,
1. INCORRECT. ALL 3 VARIABLES ARE BOUNDED:
0 <= X1, Y <= 1000
0<= X2 <= 1.
2. HOWEVER, SINCE THE INPUT SCALES ARE DIFFERENT BY A FACTOR OF THOUSAND, X1 AND Y SHOULD BE TRANSFORMED BY VIA LOGS AND/OR POWERS. E.G.,
X1n = LOG10( 1 + X1 ) / LOG10( 1001 ) ==> 0 <= X1n <= 1
SIMILARLY FOR Y
HOPE THIS HELPS.
GREG

More Answers (1)

Greg Heath
Greg Heath on 29 Jan 2012
If the target has rigid bounds, scale the data to either [0,1] or [-1,1] and use either LOGSIG or TANSIG, respectively.
Otherwise, standardize to zero-mean/unit variance and use PURELIN.
To recover the original data scale, just use the reverse tranformations.
Hope this helps.
Greg
  1 Comment
Ashikur
Ashikur on 29 Jan 2012
Hello Greg,
Thanks again for answering the question. For my case, no rigid bound, So I will use purelin or f(x) = x
1. Standardize what? the input? each input vector individually? or the whole training input matrix?
2. Do I need to do the same standardization for desired output too?
I used purlin (f(x)=x) in my output, but it seems not to converge for,
0 0 ------> 0
0 1 ------> 1000
1000 1 ----> 1
1 1 -------> 0
The codes are in below link,
https://docs.google.com/document/d/18kGXAgeVP1kOlJHZJ2KmV_mkxY61f255LQjPcr6uwnI/edit?pli=1
https://docs.google.com/document/d/13YVWHUXug6XrngQD3a52IgzPxMH2M9sgECkb7peRckA/edit?pli=1
Hope you will have some times to see the problem.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!