Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!13g2000yql.googlegroups.com!not-for-mail
From: Greg Heath <heath@alumni.brown.edu>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Backpropagation Neural network Code problem
Date: Wed, 18 Mar 2009 08:03:30 -0700 (PDT)
Organization: http://groups.google.com
Lines: 174
Message-ID: <e47cb115-6371-4a9d-a989-f1190d85a46f@13g2000yql.googlegroups.com>
References: <gpfjg6$t0n$1@fred.mathworks.com> <gpfk0u$g8$1@fred.mathworks.com> 
	<dae52b80-1b2c-436a-9dd5-d6127ab8b97b@j35g2000yqh.googlegroups.com> 
	<gpqbce$1fe$1@fred.mathworks.com>
NNTP-Posting-Host: 69.141.163.135
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1237388612 17353 127.0.0.1 (18 Mar 2009 15:03:32 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Wed, 18 Mar 2009 15:03:32 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: 13g2000yql.googlegroups.com; posting-host=69.141.163.135; 
	posting-account=mUealwkAAACvQrLWvunjg50tRAnsNtJR
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 
	5.1),gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:525879


On Mar 18, 4:29=A0am, "Adeel " <neoresearc...@gmail.com> wrote:
> http://www.mathworks.com/matlabcentral/newsreader/create_message?repl...
>
> Greg Heath <he...@alumni.brown.edu> wrote in message
> <dae52b80-1b2c-436a-9dd5-d6127ab8b...@j35g2000yqh.googlegroups.com>...
> > On Mar 17, 3:13=3DA0pm, Greg Heath <he...@alumni.brown.edu> wrote:
> > > On Mar 17, 2:40 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> > > > On Mar 16, 11:46 pm, "Adeel " <neoresearc...@gmail.com> wrote:
> > > > > 1. OK i will insert much more comments in the code.
> > > > > 2. You are right that the other layer is output layer, But I had =
two =3D
> > layers, because output layer had weights (plus bias) on them. Not to me=
ntio=3D
> > n the activation function etc.
>
> > > Two important points on terminology:
>
> > > 1. The input layer is not a neuron layer.
> > > 2. The output layer is not a hidden layer. (correct your comment in
> > > the code)
>
> > > > > 3. The name of this file is element wise, that is element by elem=
ent =3D
> > every step, Just a precaution to safe guard against any malfunction or =
mist=3D
> > ake in the matrix multiplication done by me. (Although i had created on=
e fi=3D
> > le using matrix multiplication).
>
> > > > > 4. OK i agree i did not know that. Plus i did not know what "loga=
rith=3D
> > mic sigmoid"
>
> > WHOOPS! I meant logistic sigmoid. See below
>
> > > > > is. Are u talking about bipolar sigmoid.
>
> > No. Bipolar sigmoid is a scaled tanh. See beow.
>
> > ( I have not used tanh, I had used bipolar sigmoid.). In the book
> > "Fundamentals ofneuralnetworks by laurene Fausett of pearson
> > education" They had used for both layer the bipolar sigmoid. and
> > according to the text they had converge the weight using the bipolar
> > sigmoid as an activation function on both the layers. (unless i had
> > understood it incorrectly. According to them they had solve the xor
> > problem solution in binary representation, to get converge in 3000
> > epochs.. Mine never did)
>
> > > logsig(x) =3D3D 1/(1+exp(-x))
>
> > Logistic Sigmoid
>
> > > tanh(x/2) =3D3D 2*logsig(x)-1
>
> > > You might be interested in
>
> > >http://groups.google.com/group/comp.ai.neural-nets/
> > > msg/1c33c39b186dd026?hl=3D3Den
>
> > > Note the learning times using the batch L-M algorithm.
>
> > Also
>
> >http://groups.google.com/group/comp.ai.neural-nets/browse_thread/thre...
> > 4b0a03ffcdafb?hl=3D3Den&tvc=3D3D2&q=3D3Dgreg-heath+backpropagation+tuto=
rial+
>
> > Hope this helps.
>
> > Greg
>
> Sir.
> My basic and only concern is "Why did my algo did not converge ?"

You made one or more mistakes. You may have to go through an
iteration
or two by hand.

> Is binary sigmoid is logistic sigmoid ( logsig(x) ) ?

In the last 20 years I  have never heard of a binary sigmoid
and I'm glad I haven't because the term does not make sense.
Binary implies two values whereas a sigmoid is real valued.

Did you mean bipolar sigmoid? Bipolar implies it contains
both positive and negative value.

If you have the NN Toolbox type (otherwise read the NN Toolbox
documentation on the MATLAB site)

doc logsig       % Logarithmic sigmoid
                      % logsig(n) =3D 1 / (1 + exp(-n))

doc tansig      % Hyperbolic tangent sigmoid
                     % tansig(n) =3D 2/(1+exp(-2*n))-1 =3D tanh(n)

Most of the world uses the term logistic sigmoid for logsig.
(Google on both logidtic-sigmoid and logarithmic-sigmoid).
Logsig is unipolar.

Tansig is bipolar.


> You are right. Input layer is not included in the layer count. (which i a=
lso didnot counted).

Be careful. Some authors do count it.

But i
>
> counted output layer because and hidden layer =A0(the layer next to input=
 layer) because both those
>
> layer had weight which did change in the learning process.. Secondly the =
text book also called it
>
> two layer. Another text book "Neural network. Algorithums, applications, =
and programming techniques
>
> by James A. Freeman / David M. Skapura" also called it two layer backprop=
agation network. If i had
>
> understood it incorrectly I would be greatful if u correct me.
> I think in matlab documentation they didnot count the output layer. (i wi=
ll conform that when i
>
> check that) if u r saying according to that then i agree with u.

You missed my point. In your code you called the output layer
the second hidden layer. That is incorrect.

Now that you've mentioned layer count, let me clarify:

Different authors use different ways to count layers.

Some consider layers of nodes
Some consider layers of neurons
Some consider layers of weights

Some count input layers
Some count output layers

I, on the otherhand, am always unambiguous because
I just count hidden layers.


>
> the last link that you sendhttp://groups.google.com/group/comp.ai.neural-=
nets/browse_thread/thre...
>
> c=3D3D2&q=3D3Dgreg-heath+backpropagation+tutorial+
> orhttp://groups.google.com/group/comp.ai.neural-nets/browse_thread/thre..=
.
> didnot render a page the responce was
> Topic not found
> "We're sorry, but we were unable to find the topic you were looking for. =
Perhaps the URL you clicked
>
> on is out of date or broken?

Find the 5 message comp.ai.neural-net thread

Basic Backpropagation Learning Strategy (Tutorial)

The first post is at

http://groups.google.com/group/comp.ai.neural-nets/msg/d35128ccb25c7779?hl=
=3Den

and contains a link to the thread.

Hope this helps.

Greg