Thread Subject: Neural Network - unclear what is Matlab doing there.. (help needed)

Hi,
I am looking to build a Feed forward network for OCR , I found out that there is a good example on MAtlab help, the only problem with the help is that it is too condensed, and you can't understand from the help what they really do.

My questions is :

1) In the help there is some schematic drawing of the net, and they say in the beginning :
"The hidden (first) layer has 25 neurons" but in the drawing there is only 10...
Am i'm missing something ?
What is the architecture of the net? does every neuron connect to every neuron ?

2) When you train the net,
is every column in the alphabet matrix suppose to be matched with the same index column in the target matrix ?
if this is the case, is it possible to use the same net to train it with fewer letters (e.g. smaller alphabet matrix ) but with the same target vector?
(means, to just find the A,B,C,D ... without the rest..)

Thanks
O


 

On Nov 14, 5:15 pm, "ODPR POOO" <odprt...@gmail.com> wrote:
> Hi,
> I am looking to build a Feed forward network for OCR , I found out that there is a good example on  MAtlab help, the only problem with the help is that it is too condensed, and you can't understand from the help what they really do.
>
> My questions is :
>
> 1) In the help there is some schematic drawing of the net, and they say in the beginning :
> "The hidden (first) layer has 25 neurons" but in the drawing there is only 10...
> Am i'm missing something ?
> What is the architecture of the net? does every neuron connect  to every neuron ?
>
> 2) When you train the net,
> is every column in the alphabet matrix suppose to be matched with the same index column in the target matrix ?
> if this is the case, is it possible to use the same net to train it with fewer letters (e.g. smaller alphabet matrix ) but with the same target vector?
> (means, to just find the A,B,C,D ... without the rest..)
>
> Thanks
> O

Where, exactly, is this example?

Greg

In the help of MAtlab, either write in the help search :appcr1
or write : character recognition


On Nov 16, 12:46 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 14, 5:15 pm, "ODPR POOO" <odprt...@gmail.com> wrote:
>
>
>
>
>
> > Hi,
> > I am looking to build a Feed forward network for OCR , I found out that there is a good example on  MAtlab help, the only problem with the help is that it is too condensed, and you can't understand from the help what they really do.
>
> > My questions is :
>
> > 1) In the help there is some schematic drawing of the net, and they say in the beginning :
> > "The hidden (first) layer has 25 neurons" but in the drawing there is only 10...
> > Am i'm missing something ?
> > What is the architecture of the net? does every neuron connect  to every neuron ?
>
> > 2) When you train the net,
> > is every column in the alphabet matrix suppose to be matched with the same index column in the target matrix ?
> > if this is the case, is it possible to use the same net to train it with fewer letters (e.g. smaller alphabet matrix ) but with the same target vector?
> > (means, to just find the A,B,C,D ... without the rest..)
>
> > Thanks
> > O
>
> Where, exactly, is this example?
>
> Greg

On Nov 14, 5:15 pm, "ODPR POOO" <odprt...@gmail.com> wrote:
> Hi,
> I am looking to build a Feed forward network for OCR , I found
> out that there is a good example on MAtlab help, the only
> problem with the help is that it is too condensed, and you can't
> understand from the help what they really do.
>
> My questions is :
>
> 1) In the help there is some schematic drawing of the net, and
> they say in the beginning :
> "The hidden (first) layer has 25 neurons" but in the drawing
> there is only 10... Am i'm missing something ?

 My version says 10 neurons. Your version must have a misprint.

> What is the architecture of the net?

The figure is very explicit.
The text says newff ("ff" ==> feed-forward)
The net is a Feed-Forward-Multi-Layer=Perceptron (FFMLP)

help newff
doc newff

> does every neuron connect to every neuron ?

No.

As shown in the figure, neuron outputs are only connected
to neuron inputs of the next layer (no feedback, self-layer
or skip-layer connections)


> 2) When you train the net,
> is every column in the alphabet matrix suppose to be matched
> with the same index column in the target matrix ?

Yes

> if this is the case, is it possible to use the same net to
> train it with fewer letters (e.g. smaller alphabet matrix )
> but with the same target vector?
> (means, to just find the A,B,C,D ... without the rest..)

Yes; as long as inputs and targets are matched.

However, further training with a subset after training
with the complete set may degrade performance on the
letters omitted in the additional training.

Hope this helps.

Greg

Thanks for the help! :)

Few more things though :


>
1) In the help there is some schematic drawing of the net, and
> they say in the beginning :
> "The hidden (first) layer has 25 neurons" but in the drawing
> there is only 10... Am i'm missing something ?
>> My version says 10 neurons. Your version must have a misprint.

is yours say that as well :

net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
in the hidden layer, no ?

You have : net = newff(alphabet,targets,10); ??


>
> > does every neuron connect  to every neuron ?
>
> No.

I meant that every neuron output (10 of them) in the hidden layer is
connected to every input in the last layer (26 of them )

Is that correct assumption ?


Where can I read more on which method MAtlab team used in the train
command?
Thanks!
O

On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
> Thanks for the help!  :)
>
> Few more things though :
> 1) In the help there is some schematic drawing of the net, and
> > they say in the beginning :
> > "The hidden (first) layer has 25 neurons" but in the drawing
> > there is only 10...  Am i'm missing something ?
>
> >>  My version says 10 neurons. Your version must have a misprint.
>
> is yours say that as well  :
>
> net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> in the hidden layer, no ?
>
> You have   :    net = newff(alphabet,targets,10);   ??

No. I have

BEGIN QUOTE

The hidden (first) layer has 10 neurons. This number was picked by
guesswork and experience. If the network has trouble learning, then
neurons can be added to this layer.

-----SNIP.

Initialization

The two-layer network is created with newff.

S1 = 10;
[R,Q] = size(alphabet);
[S2,Q] = size(targets);
P = alphabet;
net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');

END QUOTE


> > > does every neuron connect  to every neuron ?
>
> > No.
>
> I meant that every neuron output (10 of them) in the hidden layer   is
> connected to every input in the last layer (26 of them )
>
> Is that correct assumption ?

Yes

> Where can I read more on which method MAtlab team used in the train
> command?

help traingcx
doc traingdx

Hope this helps.

Greg

On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > Thanks for the help!  :)
>
> > Few more things though :
> > 1) In the help there is some schematic drawing of the net, and
> > > they say in the beginning :
> > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > there is only 10...  Am i'm missing something ?
>
> > >>  My version says 10 neurons. Your version must have a misprint.
>
> > is yours say that as well  :
>
> > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > in the hidden layer, no ?
>
> > You have   :    net = newff(alphabet,targets,10);   ??
>
> No. I have
>
> BEGIN QUOTE
>
> The hidden (first) layer has 10 neurons. This number was picked by
> guesswork and experience. If the network has trouble learning, then
> neurons can be added to this layer.
>
> -----SNIP.
>
> Initialization
>
> The two-layer network is created with newff.
>
> S1 = 10;
> [R,Q] = size(alphabet);
> [S2,Q] = size(targets);
> P = alphabet;
> net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> END QUOTE
>
> > > > does every neuron connect  to every neuron ?
>
> > > No.
>
> > I meant that every neuron output (10 of them) in the hidden layer   is
> > connected to every input in the last layer (26 of them )
>
> > Is that correct assumption ?
>
> Yes
>
> > Where can I read more on which method MAtlab team used in the train
> > command?
>
> help traingdx
> doc traingdx

Ignore this. I have an obsolete version. The command
you are using uses the default 'trainlm'

help trainlm
doc trainlm

Hope this helps.

Greg

On Nov 18, 1:00 am, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > Thanks for the help!  :)
>
> > Few more things though :
> > 1) In the help there is some schematic drawing of the net, and
> > > they say in the beginning :
> > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > there is only 10...  Am i'm missing something ?
>
> > >>  My version says 10 neurons. Your version must have a misprint.
>
> > is yours say that as well  :
>
> > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > in the hidden layer, no ?
>
> > You have   :    net = newff(alphabet,targets,10);   ??
>
> No. I have
>
> BEGIN QUOTE
>
> The hidden (first) layer has 10 neurons. This number was picked by
> guesswork and experience. If thenetworkhas trouble learning, then
> neurons can be added to this layer.
>
> -----SNIP.
>
> Initialization
>
> The two-layernetworkis created with newff.
>
> S1 = 10;
> [R,Q] = size(alphabet);
> [S2,Q] = size(targets);
> P = alphabet;
> net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> END QUOTE
>
> > > > does every neuron connect  to every neuron ?
>
> > > No.
>
> > I meant that every neuron output (10 of them) in the hidden layer   is
> > connected to every input in the last layer (26 of them )
>
> > Is that correct assumption ?
>
> Yes
>
> > Where can I read more on which method MAtlab team used in the train
> > command?
>
> help traingcx
> doc traingdx
>
> Hope this helps.
>
> Greg

Thanks, I am wondering how come there is big diffrences in the help,
I have the latest matlab version,
in any case thank u very much it was very helpful

On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > Thanks for the help!  :)
>
> > > Few more things though :
> > > 1) In the help there is some schematic drawing of the net, and
> > > > they say in the beginning :
> > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > there is only 10...  Am i'm missing something ?
>
> > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > is yours say that as well  :
>
> > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > in the hidden layer, no ?
>
> > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > No. I have
>
> > BEGIN QUOTE
>
> > The hidden (first) layer has 10 neurons. This number was picked by
> > guesswork and experience. If the network has trouble learning, then
> > neurons can be added to this layer.
>
> > -----SNIP.
>
> > Initialization
>
> > The two-layer network is created with newff.
>
> > S1 = 10;
> > [R,Q] = size(alphabet);
> > [S2,Q] = size(targets);
> > P = alphabet;
> > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > END QUOTE
>
> > > > > does every neuron connect  to every neuron ?
>
> > > > No.
>
> > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > connected to every input in the last layer (26 of them )
>
> > > Is that correct assumption ?
>
> > Yes
>
> > > Where can I read more on which method MAtlab team used in the train
> > > command?
>
> > help traingdx
> > doc traingdx
>
> Ignore this. I have an obsolete version. The command
> you are using uses the default 'trainlm'
>
> help trainlm
> doc trainlm
>
> Hope this helps.
>
> Greg

Also question on the training phase:
- If I train the net with matrix 4*4 for example and I always feed
the net with column 1 2 3 (which represent three different
letters ) with the same data all the time, but change the data in
the 4 column, (same letter in OCR example , but different noise leve )
is it true (or not), that the weights will change only where they need
too (e.g for the 4 column) , and won't ruin the ability of the net to
discover the other 3 ?

On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
> On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > Thanks for the help!  :)
>
> > > > Few more things though :
> > > > 1) In the help there is some schematic drawing of the net, and
> > > > > they say in the beginning :
> > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > there is only 10...  Am i'm missing something ?
>
> > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > is yours say that as well  :
>
> > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > in the hidden layer, no ?
>
> > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > No. I have
>
> > > BEGIN QUOTE
>
> > > The hidden (first) layer has 10 neurons. This number was picked by
> > > guesswork and experience. If the network has trouble learning, then
> > > neurons can be added to this layer.
>
> > > -----SNIP.
>
> > > Initialization
>
> > > The two-layer network is created with newff.
>
> > > S1 = 10;
> > > [R,Q] = size(alphabet);
> > > [S2,Q] = size(targets);
> > > P = alphabet;
> > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > END QUOTE
>
> > > > > > does every neuron connect  to every neuron ?
>
> > > > > No.
>
> > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > connected to every input in the last layer (26 of them )
>
> > > > Is that correct assumption ?
>
> > > Yes
>
> > > > Where can I read more on which method MAtlab team used in the train
> > > > command?
>
> > > help traingdx
> > > doc traingdx
>
> > Ignore this. I have an obsolete version. The command
> > you are using uses the default 'trainlm'
>
> > help trainlm
> > doc trainlm
>
> > Hope this helps.
>
> > Greg
>
> Also question on the training phase:
> -  If I train the net with matrix 4*4 for example and I always feed
> the net with column 1 2 3 (which represent three different
> letters  )  with the same data all the time, but change the data in
> the 4 column, (same letter in OCR example , but different noise leve )
> is it true (or not), that the weights will change only where they need
> too (e.g for the 4 column)  , and won't ruin the ability of the net to
> discover the other 3

I don't see a problem with that scenario.

However, the devil is in the details.

Hope this helps.

Greg

On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > Thanks for the help!  :)
>
> > > > > Few more things though :
> > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > they say in the beginning :
> > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > is yours say that as well  :
>
> > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > in the hidden layer, no ?
>
> > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > No. I have
>
> > > > BEGIN QUOTE
>
> > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > guesswork and experience. If the network has trouble learning, then
> > > > neurons can be added to this layer.
>
> > > > -----SNIP.
>
> > > > Initialization
>
> > > > The two-layer network is created with newff.
>
> > > > S1 = 10;
> > > > [R,Q] = size(alphabet);
> > > > [S2,Q] = size(targets);
> > > > P = alphabet;
> > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > END QUOTE
>
> > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > No.
>
> > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > connected to every input in the last layer (26 of them )
>
> > > > > Is that correct assumption ?
>
> > > > Yes
>
> > > > > Where can I read more on which method MAtlab team used in the train
> > > > > command?
>
> > > > help traingdx
> > > > doc traingdx
>
> > > Ignore this. I have an obsolete version. The command
> > > you are using uses the default 'trainlm'
>
> > > help trainlm
> > > doc trainlm
>
> > > Hope this helps.
>
> > > Greg
>
> > Also question on the training phase:
> > -  If I train the net with matrix 4*4 for example and I always feed
> > the net with column 1 2 3 (which represent three different
> > letters  )  with the same data all the time, but change the data in
> > the 4 column, (same letter in OCR example , but different noise leve )
> > is it true (or not), that the weights will change only where they need
> > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > discover the other 3
>
> I don't see a problem with that scenario.
>
> However, the devil is in the details.
>
> Hope this helps.
>
> Greg

Well Greg thank you :)
I will work on this net for few days now..

On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
> On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > Thanks for the help!  :)
>
> > > > > > Few more things though :
> > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > they say in the beginning :
> > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > is yours say that as well  :
>
> > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > in the hidden layer, no ?
>
> > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > No. I have
>
> > > > > BEGIN QUOTE
>
> > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > guesswork and experience. If the network has trouble learning, then
> > > > > neurons can be added to this layer.
>
> > > > > -----SNIP.
>
> > > > > Initialization
>
> > > > > The two-layer network is created with newff.
>
> > > > > S1 = 10;
> > > > > [R,Q] = size(alphabet);
> > > > > [S2,Q] = size(targets);
> > > > > P = alphabet;
> > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > END QUOTE
>
> > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > No.
>
> > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > Is that correct assumption ?
>
> > > > > Yes
>
> > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > command?
>
> > > > > help traingdx
> > > > > doc traingdx
>
> > > > Ignore this. I have an obsolete version. The command
> > > > you are using uses the default 'trainlm'
>
> > > > help trainlm
> > > > doc trainlm
>
> > > > Hope this helps.
>
> > > > Greg
>
> > > Also question on the training phase:
> > > -  If I train the net with matrix 4*4 for example and I always feed
> > > the net with column 1 2 3 (which represent three different
> > > letters  )  with the same data all the time, but change the data in
> > > the 4 column, (same letter in OCR example , but different noise leve )
> > > is it true (or not), that the weights will change only where they need
> > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > discover the other 3
>
> > I don't see a problem with that scenario.
>
> > However, the devil is in the details.
>
> > Hope this helps.
>
> > Greg
>
> Well Greg thank you :)
> I will work on this net for few days now..

well after playing with the nets for a while I stumbled upon something
VERY unclear,could be a bug from Matlab, or something I am doing
wrong, I have no idea. the scenario goes like this :

I type :

----------
net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
'logsig'},'trainlm');
 [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
----------

The net is working nice, and finding stuff.



But when I write the same line again :

net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
'logsig'},'trainlm');
 [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);

The net can't find anything anymore... (e.g. gives 0.5 on all the
exits.. )

How can that be?

On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
> On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > Thanks for the help!  :)
>
> > > > > > > Few more things though :
> > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > they say in the beginning :
> > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > is yours say that as well  :
>
> > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > in the hidden layer, no ?
>
> > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > No. I have
>
> > > > > > BEGIN QUOTE
>
> > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > neurons can be added to this layer.
>
> > > > > > -----SNIP.
>
> > > > > > Initialization
>
> > > > > > The two-layer network is created with newff.
>
> > > > > > S1 = 10;
> > > > > > [R,Q] = size(alphabet);
> > > > > > [S2,Q] = size(targets);
> > > > > > P = alphabet;
> > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > END QUOTE
>
> > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > No.
>
> > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > Is that correct assumption ?
>
> > > > > > Yes
>
> > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > command?
>
> > > > > > help traingdx
> > > > > > doc traingdx
>
> > > > > Ignore this. I have an obsolete version. The command
> > > > > you are using uses the default 'trainlm'
>
> > > > > help trainlm
> > > > > doc trainlm
>
> > > > > Hope this helps.
>
> > > > > Greg
>
> > > > Also question on the training phase:
> > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > the net with column 1 2 3 (which represent three different
> > > > letters  )  with the same data all the time, but change the data in
> > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > is it true (or not), that the weights will change only where they need
> > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > discover the other 3
>
> > > I don't see a problem with that scenario.
>
> > > However, the devil is in the details.
>
> > > Hope this helps.
>
> > > Greg
>
> > Well Greg thank you :)
> > I will work on this net for few days now..
>
> well after playing with the nets for a while I stumbled upon something
> VERY unclear,could be a bug from Matlab, or  something I am doing
> wrong,  I have no idea. the scenario goes like this :
>
> I type :
>
> ----------
> net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> 'logsig'},'trainlm');
>  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> ----------
>
> The net is working nice, and finding stuff.
>
> But when I write  the same line again :

What does that mean? Is this later in the same program
or are you rerunning the program.
>
> net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> 'logsig'},'trainlm');
>  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> The net can't find anything anymore... (e.g. gives 0.5 on all the
> exits.. )
>
> How can that be

net = newff(...)
always creates a new net with new initial weights.
Perhaps your 1st training converged to a low local
minimum but your 2nd did not.

Hope this helps.

gREG

On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > Thanks for the help!  :)
>
> > > > > > > > Few more things though :
> > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > they say in the beginning :
> > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > is yours say that as well  :
>
> > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > in the hidden layer, no ?
>
> > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > No. I have
>
> > > > > > > BEGIN QUOTE
>
> > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > neurons can be added to this layer.
>
> > > > > > > -----SNIP.
>
> > > > > > > Initialization
>
> > > > > > > The two-layer network is created with newff.
>
> > > > > > > S1 = 10;
> > > > > > > [R,Q] = size(alphabet);
> > > > > > > [S2,Q] = size(targets);
> > > > > > > P = alphabet;
> > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > END QUOTE
>
> > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > No.
>
> > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > Is that correct assumption ?
>
> > > > > > > Yes
>
> > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > command?
>
> > > > > > > help traingdx
> > > > > > > doc traingdx
>
> > > > > > Ignore this. I have an obsolete version. The command
> > > > > > you are using uses the default 'trainlm'
>
> > > > > > help trainlm
> > > > > > doc trainlm
>
> > > > > > Hope this helps.
>
> > > > > > Greg
>
> > > > > Also question on the training phase:
> > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > the net with column 1 2 3 (which represent three different
> > > > > letters  )  with the same data all the time, but change the data in
> > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > is it true (or not), that the weights will change only where they need
> > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > discover the other 3
>
> > > > I don't see a problem with that scenario.
>
> > > > However, the devil is in the details.
>
> > > > Hope this helps.
>
> > > > Greg
>
> > > Well Greg thank you :)
> > > I will work on this net for few days now..
>
> > well after playing with the nets for a while I stumbled upon something
> > VERY unclear,could be a bug from Matlab, or  something I am doing
> > wrong,  I have no idea. the scenario goes like this :
>
> > I type :
>
> > ----------
> > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > 'logsig'},'trainlm');
> >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > ----------
>
> > The net is working nice, and finding stuff.
>
> > But when I write  the same line again :
>
> What does that mean? Is this later in the same program
> or are you rerunning the program.
>
>
>
> > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > 'logsig'},'trainlm');
> >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > exits.. )
>
> > How can that be
>
> net = newff(...)
> always creates a new net with new initial weights.
> Perhaps your 1st training converged to a low local
> minimum but your 2nd did not.
>
> Hope this helps.
>
> gREG

??

That means that training a net isn't deterministic method? then how
one can design a net? if some times it work ok, and the next time when
you want to build the same net, it doesn't work ?

How can you count on this method?

O

On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
> On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > Few more things though :
> > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > they say in the beginning :
> > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > is yours say that as well  :
>
> > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > No. I have
>
> > > > > > > > BEGIN QUOTE
>
> > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > neurons can be added to this layer.
>
> > > > > > > > -----SNIP.
>
> > > > > > > > Initialization
>
> > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > S1 = 10;
> > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > [S2,Q] = size(targets);
> > > > > > > > P = alphabet;
> > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > END QUOTE
>
> > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > No.
>
> > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > Is that correct assumption ?
>
> > > > > > > > Yes
>
> > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > command?
>
> > > > > > > > help traingdx
> > > > > > > > doc traingdx
>
> > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > help trainlm
> > > > > > > doc trainlm
>
> > > > > > > Hope this helps.
>
> > > > > > > Greg
>
> > > > > > Also question on the training phase:
> > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > the net with column 1 2 3 (which represent three different
> > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > is it true (or not), that the weights will change only where they need
> > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > discover the other 3
>
> > > > > I don't see a problem with that scenario.
>
> > > > > However, the devil is in the details.
>
> > > > > Hope this helps.
>
> > > > > Greg
>
> > > > Well Greg thank you :)
> > > > I will work on this net for few days now..
>
> > > well after playing with the nets for a while I stumbled upon something
> > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > wrong,  I have no idea. the scenario goes like this :
>
> > > I type :
>
> > > ----------
> > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > 'logsig'},'trainlm');
> > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > ----------
>
> > > The net is working nice, and finding stuff.
>
> > > But when I write  the same line again :
>
> > What does that mean? Is this later in the same program
> > or are you rerunning the program.
>
> > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > 'logsig'},'trainlm');
> > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > exits.. )
>
> > > How can that be
>
> > net = newff(...)
> > always creates a new net with new initial weights.
> > Perhaps your 1st training converged to a low local
> > minimum but your 2nd did not.
>
> > Hope this helps.
>
> > gREG
>
> ??
>
> That means that training a net isn't deterministic  method? then how
> one can design a net? if some times it work ok, and the next time when
> you want to build the same net, it doesn't work ?
>
> How can you count on this method?

One of two ways

1. Before you call newff reset the random number generator
    to a fixed initial state of your choice.

myinitialstate = mod(myfavoriteinteger,2^32-1)
rand('state',myinitialstate)
net = newff(...)

2. After you call newff reset the initial weights to a fixed set
    of your choice.

net = newff(..)
net.IW{1,1} = IW0;
net.LW(2.1} = LW0;
net.b{1} = b1;
net.b{2} = b2;

I favor method 1.

Hope this helps.

Greg

On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > Few more things though :
> > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > No. I have
>
> > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > -----SNIP.
>
> > > > > > > > > Initialization
>
> > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > S1 = 10;
> > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > P = alphabet;
> > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > END QUOTE
>
> > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > No.
>
> > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > Yes
>
> > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > command?
>
> > > > > > > > > help traingdx
> > > > > > > > > doc traingdx
>
> > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > help trainlm
> > > > > > > > doc trainlm
>
> > > > > > > > Hope this helps.
>
> > > > > > > > Greg
>
> > > > > > > Also question on the training phase:
> > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > discover the other 3
>
> > > > > > I don't see a problem with that scenario.
>
> > > > > > However, the devil is in the details.
>
> > > > > > Hope this helps.
>
> > > > > > Greg
>
> > > > > Well Greg thank you :)
> > > > > I will work on this net for few days now..
>
> > > > well after playing with the nets for a while I stumbled upon something
> > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > I type :
>
> > > > ----------
> > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > 'logsig'},'trainlm');
> > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > ----------
>
> > > > The net is working nice, and finding stuff.
>
> > > > But when I write  the same line again :
>
> > > What does that mean? Is this later in the same program
> > > or are you rerunning the program.
>
> > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > 'logsig'},'trainlm');
> > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > exits.. )
>
> > > > How can that be
>
> > > net = newff(...)
> > > always creates a new net with new initial weights.
> > > Perhaps your 1st training converged to a low local
> > > minimum but your 2nd did not.
>
> > > Hope this helps.
>
> > > gREG
>
> > ??
>
> > That means that training a net isn't deterministic  method? then how
> > one can design a net? if some times it work ok, and the next time when
> > you want to build the same net, it doesn't work ?
>
> > How can you count on this method?
>
> One of two ways
>
> 1. Before you call newff reset the random number generator
>     to a fixed initial state of your choice.
>
> myinitialstate = mod(myfavoriteinteger,2^32-1)
> rand('state',myinitialstate)
> net = newff(...)
>
> 2. After you call newff reset the initial weights to a fixed set
>     of your choice.
>
> net = newff(..)
> net.IW{1,1} = IW0;
> net.LW(2.1} = LW0;
> net.b{1} = b1;
> net.b{2} = b2;
>
> I favor method 1.
>
> Hope this helps.
>
> Greg


Can I expect the net to recognize with full ( or with high degree of
certainty ) the train set letters?
(e.g. I will give it some of the letters that it trained on, and will
receive high value on that node.)

I want to know if this is a good check for the net ability..


Thanks
O

On Nov 29, 2:08 pm, Eizo <odpe...@gmail.com> wrote:
> On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > > Few more things though :
> > > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > > No. I have
>
> > > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > > -----SNIP.
>
> > > > > > > > > > Initialization
>
> > > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > > S1 = 10;
> > > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > > P = alphabet;
> > > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > > END QUOTE
>
> > > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > > No.
>
> > > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > > Yes
>
> > > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > > command?
>
> > > > > > > > > > help traingdx
> > > > > > > > > > doc traingdx
>
> > > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > > help trainlm
> > > > > > > > > doc trainlm
>
> > > > > > > > > Hope this helps.
>
> > > > > > > > > Greg
>
> > > > > > > > Also question on the training phase:
> > > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > > discover the other 3
>
> > > > > > > I don't see a problem with that scenario.
>
> > > > > > > However, the devil is in the details.
>
> > > > > > > Hope this helps.
>
> > > > > > > Greg
>
> > > > > > Well Greg thank you :)
> > > > > > I will work on this net for few days now..
>
> > > > > well after playing with the nets for a while I stumbled upon something
> > > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > > I type :
>
> > > > > ----------
> > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > 'logsig'},'trainlm');
> > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > > ----------
>
> > > > > The net is working nice, and finding stuff.
>
> > > > > But when I write  the same line again :
>
> > > > What does that mean? Is this later in the same program
> > > > or are you rerunning the program.
>
> > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > 'logsig'},'trainlm');
> > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > > exits.. )
>
> > > > > How can that be
>
> > > > net = newff(...)
> > > > always creates a new net with new initial weights.
> > > > Perhaps your 1st training converged to a low local
> > > > minimum but your 2nd did not.
>
> > > > Hope this helps.
>
> > > > gREG
>
> > > ??
>
> > > That means that training a net isn't deterministic  method? then how
> > > one can design a net? if some times it work ok, and the next time when
> > > you want to build the same net, it doesn't work ?
>
> > > How can you count on this method?
>
> > One of two ways
>
> > 1. Before you call newff reset the random number generator
> >     to a fixed initial state of your choice.
>
> > myinitialstate = mod(myfavoriteinteger,2^32-1)
> > rand('state',myinitialstate)
> > net = newff(...)
>
> > 2. After you call newff reset the initial weights to a fixed set
> >     of your choice.
>
> > net = newff(..)
> > net.IW{1,1} = IW0;
> > net.LW(2.1} = LW0;
> > net.b{1} = b1;
> > net.b{2} = b2;
>
> > I favor method 1.
>
> > Hope this helps.
>
> > Greg
>
> Can I expect the net to recognize with full ( or with high degree  of
> certainty ) the train set letters?
> (e.g. I will give it some of the letters that it trained on, and will
> receive high value on that node.)
>
> I want to know if this is a good check for the net ability..

I consider it necessary but not sufficient.

You need a sufficiently large nondesign sample to estimate
generalization error.

Hope this helps.

Greg

On Nov 30, 6:01 am, Greg Heath <he...@alumni.brown.edu> wrote:
> On Nov 29, 2:08 pm, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > > > Few more things though :
> > > > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > > > No. I have
>
> > > > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > > > -----SNIP.
>
> > > > > > > > > > > Initialization
>
> > > > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > > > S1 = 10;
> > > > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > > > P = alphabet;
> > > > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > > > END QUOTE
>
> > > > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > > > No.
>
> > > > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > > > Yes
>
> > > > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > > > command?
>
> > > > > > > > > > > help traingdx
> > > > > > > > > > > doc traingdx
>
> > > > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > > > help trainlm
> > > > > > > > > > doc trainlm
>
> > > > > > > > > > Hope this helps.
>
> > > > > > > > > > Greg
>
> > > > > > > > > Also question on the training phase:
> > > > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > > > discover the other 3
>
> > > > > > > > I don't see a problem with that scenario.
>
> > > > > > > > However, the devil is in the details.
>
> > > > > > > > Hope this helps.
>
> > > > > > > > Greg
>
> > > > > > > Well Greg thank you :)
> > > > > > > I will work on this net for few days now..
>
> > > > > > well after playing with the nets for a while I stumbled upon something
> > > > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > > > I type :
>
> > > > > > ----------
> > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > 'logsig'},'trainlm');
> > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > > > ----------
>
> > > > > > The net is working nice, and finding stuff.
>
> > > > > > But when I write  the same line again :
>
> > > > > What does that mean? Is this later in the same program
> > > > > or are you rerunning the program.
>
> > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > 'logsig'},'trainlm');
> > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > > > exits.. )
>
> > > > > > How can that be
>
> > > > > net = newff(...)
> > > > > always creates a new net with new initial weights.
> > > > > Perhaps your 1st training converged to a low local
> > > > > minimum but your 2nd did not.
>
> > > > > Hope this helps.
>
> > > > > gREG
>
> > > > ??
>
> > > > That means that training a net isn't deterministic  method? then how
> > > > one can design a net? if some times it work ok, and the next time when
> > > > you want to build the same net, it doesn't work ?
>
> > > > How can you count on this method?
>
> > > One of two ways
>
> > > 1. Before you call newff reset the random number generator
> > >     to a fixed initial state of your choice.
>
> > > myinitialstate = mod(myfavoriteinteger,2^32-1)
> > > rand('state',myinitialstate)
> > > net = newff(...)
>
> > > 2. After you call newff reset the initial weights to a fixed set
> > >     of your choice.
>
> > > net = newff(..)
> > > net.IW{1,1} = IW0;
> > > net.LW(2.1} = LW0;
> > > net.b{1} = b1;
> > > net.b{2} = b2;
>
> > > I favor method 1.
>
> > > Hope this helps.
>
> > > Greg
>
> > Can I expect the net to recognize with full ( or with high degree  of
> > certainty ) the train set letters?
> > (e.g. I will give it some of the letters that it trained on, and will
> > receive high value on that node.)
>
> > I want to know if this is a good check for the net ability..
>
> I consider it necessary but not sufficient.
>
> You need a sufficiently large nondesign sample to estimate
> generalization error.
>
> Hope this helps.
>
> Greg

- When it is better to use a logsig as the output layer instead of
the regular linear?

On Dec 5, 2:23 pm, Eizo <odpe...@gmail.com> wrote:
> On Nov 30, 6:01 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > On Nov 29, 2:08 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > > > > Few more things though :
> > > > > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > > > > No. I have
>
> > > > > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > > > > -----SNIP.
>
> > > > > > > > > > > > Initialization
>
> > > > > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > > > > S1 = 10;
> > > > > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > > > > P = alphabet;
> > > > > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > > > > END QUOTE
>
> > > > > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > > > > No.
>
> > > > > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > > > > Yes
>
> > > > > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > > > > command?
>
> > > > > > > > > > > > help traingdx
> > > > > > > > > > > > doc traingdx
>
> > > > > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > > > > help trainlm
> > > > > > > > > > > doc trainlm
>
> > > > > > > > > > > Hope this helps.
>
> > > > > > > > > > > Greg
>
> > > > > > > > > > Also question on the training phase:
> > > > > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > > > > discover the other 3
>
> > > > > > > > > I don't see a problem with that scenario.
>
> > > > > > > > > However, the devil is in the details.
>
> > > > > > > > > Hope this helps.
>
> > > > > > > > > Greg
>
> > > > > > > > Well Greg thank you :)
> > > > > > > > I will work on this net for few days now..
>
> > > > > > > well after playing with the nets for a while I stumbled upon something
> > > > > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > > > > I type :
>
> > > > > > > ----------
> > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > 'logsig'},'trainlm');
> > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > > > > ----------
>
> > > > > > > The net is working nice, and finding stuff.
>
> > > > > > > But when I write  the same line again :
>
> > > > > > What does that mean? Is this later in the same program
> > > > > > or are you rerunning the program.
>
> > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > 'logsig'},'trainlm');
> > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > > > > exits.. )
>
> > > > > > > How can that be
>
> > > > > > net = newff(...)
> > > > > > always creates a new net with new initial weights.
> > > > > > Perhaps your 1st training converged to a low local
> > > > > > minimum but your 2nd did not.
>
> > > > > > Hope this helps.
>
> > > > > > gREG
>
> > > > > ??
>
> > > > > That means that training a net isn't deterministic  method? then how
> > > > > one can design a net? if some times it work ok, and the next time when
> > > > > you want to build the same net, it doesn't work ?
>
> > > > > How can you count on this method?
>
> > > > One of two ways
>
> > > > 1. Before you call newff reset the random number generator
> > > >     to a fixed initial state of your choice.
>
> > > > myinitialstate = mod(myfavoriteinteger,2^32-1)
> > > > rand('state',myinitialstate)
> > > > net = newff(...)
>
> > > > 2. After you call newff reset the initial weights to a fixed set
> > > >     of your choice.
>
> > > > net = newff(..)
> > > > net.IW{1,1} = IW0;
> > > > net.LW(2.1} = LW0;
> > > > net.b{1} = b1;
> > > > net.b{2} = b2;
>
> > > > I favor method 1.
>
> > > > Hope this helps.
>
> > > > Greg
>
> > > Can I expect the net to recognize with full ( or with high degree  of
> > > certainty ) the train set letters?
> > > (e.g. I will give it some of the letters that it trained on, and will
> > > receive high value on that node.)
>
> > > I want to know if this is a good check for the net ability..
>
> > I consider it necessary but not sufficient.
>
> > You need a sufficiently large nondesign sample to estimate
> > generalization error.
>
> > Hope this helps.
>
> > Greg
>
> -  When it is better to use a logsig as the output layer instead of
> the  regular linear

When outputs are restricted to [0,1]. For example, a classifier
with columns of the unit matrix for targets.

See my post on pretraining advice

Hope this helps.

Greg

On Dec 6, 8:51 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> On Dec 5, 2:23 pm, Eizo <odpe...@gmail.com> wrote:
>
>
>
>
>
> > On Nov 30, 6:01 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > On Nov 29, 2:08 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > > > > > Few more things though :
> > > > > > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > > > > > No. I have
>
> > > > > > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > > > > > -----SNIP.
>
> > > > > > > > > > > > > Initialization
>
> > > > > > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > > > > > S1 = 10;
> > > > > > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > > > > > P = alphabet;
> > > > > > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > > > > > END QUOTE
>
> > > > > > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > > > > > No.
>
> > > > > > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > > > > > Yes
>
> > > > > > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > > > > > command?
>
> > > > > > > > > > > > > help traingdx
> > > > > > > > > > > > > doc traingdx
>
> > > > > > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > > > > > help trainlm
> > > > > > > > > > > > doc trainlm
>
> > > > > > > > > > > > Hope this helps.
>
> > > > > > > > > > > > Greg
>
> > > > > > > > > > > Also question on the training phase:
> > > > > > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > > > > > discover the other 3
>
> > > > > > > > > > I don't see a problem with that scenario.
>
> > > > > > > > > > However, the devil is in the details.
>
> > > > > > > > > > Hope this helps.
>
> > > > > > > > > > Greg
>
> > > > > > > > > Well Greg thank you :)
> > > > > > > > > I will work on this net for few days now..
>
> > > > > > > > well after playing with the nets for a while I stumbled upon something
> > > > > > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > > > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > > > > > I type :
>
> > > > > > > > ----------
> > > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > > 'logsig'},'trainlm');
> > > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > > > > > ----------
>
> > > > > > > > The net is working nice, and finding stuff.
>
> > > > > > > > But when I write  the same line again :
>
> > > > > > > What does that mean? Is this later in the same program
> > > > > > > or are you rerunning the program.
>
> > > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > > 'logsig'},'trainlm');
> > > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > > > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > > > > > exits.. )
>
> > > > > > > > How can that be
>
> > > > > > > net = newff(...)
> > > > > > > always creates a new net with new initial weights.
> > > > > > > Perhaps your 1st training converged to a low local
> > > > > > > minimum but your 2nd did not.
>
> > > > > > > Hope this helps.
>
> > > > > > > gREG
>
> > > > > > ??
>
> > > > > > That means that training a net isn't deterministic  method? then how
> > > > > > one can design a net? if some times it work ok, and the next time when
> > > > > > you want to build the same net, it doesn't work ?
>
> > > > > > How can you count on this method?
>
> > > > > One of two ways
>
> > > > > 1. Before you call newff reset the random number generator
> > > > >     to a fixed initial state of your choice.
>
> > > > > myinitialstate = mod(myfavoriteinteger,2^32-1)
> > > > > rand('state',myinitialstate)
> > > > > net = newff(...)
>
> > > > > 2. After you call newff reset the initial weights to a fixed set
> > > > >     of your choice.
>
> > > > > net = newff(..)
> > > > > net.IW{1,1} = IW0;
> > > > > net.LW(2.1} = LW0;
> > > > > net.b{1} = b1;
> > > > > net.b{2} = b2;
>
> > > > > I favor method 1.
>
> > > > > Hope this helps.
>
> > > > > Greg
>
> > > > Can I expect the net to recognize with full ( or with high degree  of
> > > > certainty ) the train set letters?
> > > > (e.g. I will give it some of the letters that it trained on, and will
> > > > receive high value on that node.)
>
> > > > I want to know if this is a good check for the net ability..
>
> > > I consider it necessary but not sufficient.
>
> > > You need a sufficiently large nondesign sample to estimate
> > > generalization error.
>
> > > Hope this helps.
>
> > > Greg
>
> > -  When it is better to use a logsig as the output layer instead of
> > the  regular linear
>
> When outputs are restricted to [0,1]. For example, a classifier
> with columns of the unit matrix for targets.
>
> See my post on pretraining advice
>
> Hope this helps.
>
> Greg

I tried several neural nets, and decided (after consulting people) to
try new method which called SVM.
I have few questions in that matter as well, and I would be glad if
you can help.
I opened new thread on that.

http://groups.google.com/group/comp.soft-sys.matlab/browse_thread/thread/2502547ec8ba3e19#

On Dec 17, 12:07 pm, Eizo <odpe...@gmail.com> wrote:
> On Dec 6, 8:51 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
>
>
>
>
> > On Dec 5, 2:23 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > On Nov 30, 6:01 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > On Nov 29, 2:08 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > On Nov 24, 9:38 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > On Nov 24, 6:00 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > On Nov 23, 10:36 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > On Nov 21, 7:52 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > On Nov 19, 10:11 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > On Nov 19, 3:03 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > On Nov 18, 4:48 am, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > > On Nov 18, 1:06 am, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > > > On Nov 17, 6:00 pm, Greg Heath <he...@alumni.brown.edu> wrote:
>
> > > > > > > > > > > > > > On Nov 17, 2:20 pm, Eizo <odpe...@gmail.com> wrote:
>
> > > > > > > > > > > > > > > Thanks for the help!  :)
>
> > > > > > > > > > > > > > > Few more things though :
> > > > > > > > > > > > > > > 1) In the help there is some schematic drawing of the net, and
> > > > > > > > > > > > > > > > they say in the beginning :
> > > > > > > > > > > > > > > > "The hidden (first) layer has 25 neurons" but in the drawing
> > > > > > > > > > > > > > > > there is only 10...  Am i'm missing something ?
>
> > > > > > > > > > > > > > > >>  My version says 10 neurons. Your version must have a misprint.
>
> > > > > > > > > > > > > > > is yours say that as well  :
>
> > > > > > > > > > > > > > > net = newff(alphabet,targets,25); <--- this clearly means 25 neurons
> > > > > > > > > > > > > > > in the hidden layer, no ?
>
> > > > > > > > > > > > > > > You have   :    net = newff(alphabet,targets,10);   ??
>
> > > > > > > > > > > > > > No. I have
>
> > > > > > > > > > > > > > BEGIN QUOTE
>
> > > > > > > > > > > > > > The hidden (first) layer has 10 neurons. This number was picked by
> > > > > > > > > > > > > > guesswork and experience. If the network has trouble learning, then
> > > > > > > > > > > > > > neurons can be added to this layer.
>
> > > > > > > > > > > > > > -----SNIP.
>
> > > > > > > > > > > > > > Initialization
>
> > > > > > > > > > > > > > The two-layer network is created with newff.
>
> > > > > > > > > > > > > > S1 = 10;
> > > > > > > > > > > > > > [R,Q] = size(alphabet);
> > > > > > > > > > > > > > [S2,Q] = size(targets);
> > > > > > > > > > > > > > P = alphabet;
> > > > > > > > > > > > > > net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');
>
> > > > > > > > > > > > > > END QUOTE
>
> > > > > > > > > > > > > > > > > does every neuron connect  to every neuron ?
>
> > > > > > > > > > > > > > > > No.
>
> > > > > > > > > > > > > > > I meant that every neuron output (10 of them) in the hidden layer   is
> > > > > > > > > > > > > > > connected to every input in the last layer (26 of them )
>
> > > > > > > > > > > > > > > Is that correct assumption ?
>
> > > > > > > > > > > > > > Yes
>
> > > > > > > > > > > > > > > Where can I read more on which method MAtlab team used in the train
> > > > > > > > > > > > > > > command?
>
> > > > > > > > > > > > > > help traingdx
> > > > > > > > > > > > > > doc traingdx
>
> > > > > > > > > > > > > Ignore this. I have an obsolete version. The command
> > > > > > > > > > > > > you are using uses the default 'trainlm'
>
> > > > > > > > > > > > > help trainlm
> > > > > > > > > > > > > doc trainlm
>
> > > > > > > > > > > > > Hope this helps.
>
> > > > > > > > > > > > > Greg
>
> > > > > > > > > > > > Also question on the training phase:
> > > > > > > > > > > > -  If I train the net with matrix 4*4 for example and I always feed
> > > > > > > > > > > > the net with column 1 2 3 (which represent three different
> > > > > > > > > > > > letters  )  with the same data all the time, but change the data in
> > > > > > > > > > > > the 4 column, (same letter in OCR example , but different noise leve )
> > > > > > > > > > > > is it true (or not), that the weights will change only where they need
> > > > > > > > > > > > too (e.g for the 4 column)  , and won't ruin the ability of the net to
> > > > > > > > > > > > discover the other 3
>
> > > > > > > > > > > I don't see a problem with that scenario.
>
> > > > > > > > > > > However, the devil is in the details.
>
> > > > > > > > > > > Hope this helps.
>
> > > > > > > > > > > Greg
>
> > > > > > > > > > Well Greg thank you :)
> > > > > > > > > > I will work on this net for few days now..
>
> > > > > > > > > well after playing with the nets for a while I stumbled upon something
> > > > > > > > > VERY unclear,could be a bug from Matlab, or  something I am doing
> > > > > > > > > wrong,  I have no idea. the scenario goes like this :
>
> > > > > > > > > I type :
>
> > > > > > > > > ----------
> > > > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > > > 'logsig'},'trainlm');
> > > > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
> > > > > > > > > ----------
>
> > > > > > > > > The net is working nice, and finding stuff.
>
> > > > > > > > > But when I write  the same line again :
>
> > > > > > > > What does that mean? Is this later in the same program
> > > > > > > > or are you rerunning the program.
>
> > > > > > > > > net_logsig = newff(matrix_for_net,targets_heb2,5,{'logsig'
> > > > > > > > > 'logsig'},'trainlm');
> > > > > > > > >  [new_net2,tr] = train(net_logsig,matrix_for_net,targets_heb2);
>
> > > > > > > > > The net can't find anything anymore... (e.g. gives 0.5 on all the
> > > > > > > > > exits.. )
>
> > > > > > > > > How can that be
>
> > > > > > > > net = newff(...)
> > > > > > > > always creates a new net with new initial weights.
> > > > > > > > Perhaps your 1st training converged to a low local
> > > > > > > > minimum but your 2nd did not.
>
> > > > > > > > Hope this helps.
>
> > > > > > > > gREG
>
> > > > > > > ??
>
> > > > > > > That means that training a net isn't deterministic  method? then how
> > > > > > > one can design a net? if some times it work ok, and the next time when
> > > > > > > you want to build the same net, it doesn't work ?
>
> > > > > > > How can you count on this method?
>
> > > > > > One of two ways
>
> > > > > > 1. Before you call newff reset the random number generator
> > > > > >     to a fixed initial state of your choice.
>
> > > > > > myinitialstate = mod(myfavoriteinteger,2^32-1)
> > > > > > rand('state',myinitialstate)
> > > > > > net = newff(...)
>
> > > > > > 2. After you call newff reset the initial weights to a fixed set
> > > > > >     of your choice.
>
> > > > > > net = newff(..)
> > > > > > net.IW{1,1} = IW0;
> > > > > > net.LW(2.1} = LW0;
> > > > > > net.b{1} = b1;
> > > > > > net.b{2} = b2;
>
> > > > > > I favor method 1.
>
> > > > > > Hope this helps.
>
> > > > > > Greg
>
> > > > > Can I expect the net to recognize with full ( or with high degree  of
> > > > > certainty ) the train set letters?
> > > > > (e.g. I will give it some of the letters that it trained on, and will
> > > > > receive high value on that node.)
>
> > > > > I want to know if this is a good check for the net ability..
>
> > > > I consider it necessary but not sufficient.
>
> > > > You need a sufficiently large nondesign sample to estimate
> > > > generalization error.
>
> > > > Hope this helps.
>
> > > > Greg
>
> > > -  When it is better to use a logsig as the output layer instead of
> > > the  regular linear
>
> > When outputs are restricted to [0,1]. For example, a classifier
> > with columns of the unit matrix for targets.
>
> > See my post on pretraining advice
>
> > Hope this helps.
>
> > Greg
>
> I tried several neural nets, and decided (after consulting people) to
> try new method which called SVM.
> I have few questions in that matter as well, and I would be glad if
> you can help.
> I opened new thread on that.
>
> http://groups.google.com/group/comp.soft-sys.matlab/browse_thread/thr...- Hide quoted

Sorry, my MATLAB 6.5 doesn't have SVM.

Greg