Path: news.mathworks.com!newsfeed-00.mathworks.com!nlpi057.nbdc.sbc.com!prodigy.net!news.glorb.com!postnews.google.com!26g2000hsk.googlegroups.com!not-for-mail
From: Greg Heath <heath@alumni.brown.edu>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Neural Networks Cant Generalize..Results error in New Data
Date: Thu, 11 Sep 2008 00:41:11 -0700 (PDT)
Organization: http://groups.google.com
Lines: 153
Message-ID: <8436a506-ab26-4400-b349-3e2649bd757c@26g2000hsk.googlegroups.com>
References: <g9ole2$ins$1@fred.mathworks.com>
NNTP-Posting-Host: 172.132.162.251
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1221118871 17926 127.0.0.1 (11 Sep 2008 07:41:11 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Thu, 11 Sep 2008 07:41:11 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: 26g2000hsk.googlegroups.com; posting-host=172.132.162.251; 
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 
Xref: news.mathworks.com comp.soft-sys.matlab:489719



On Sep 4, 8:48=A0am, "zaheer ahmad" <ahmad.zah...@yah00000.com> wrote:
> Hi all of you.
> i got a problem inNeural=A0Networks.my net doesnt produce
> required results when new data( test ) data is applied.it is
> 100% good in memorization..im not what =A0is the problem, can
> any one help/suggest what is the actual problem...the code
> goes as below:
>
> clear;clc;
>
> % SET CHARACTERS:
> alphabet =3DAlpha4Train();
> targets=3DTargetSet();%eye(23);%

Is the empty bracket notation valid?...I've never seen it before.

> [Sa,Qa] =3D size(alphabet);
> [S2,Q] =3D size(targets);

Test to make sure that Q =3D Qa

> ValidatingChar=3DAlpha4Test();
> TestMem=3Dalphabet(:,77);
> TestGen1=3DValidatingChar(:,1);
> TestGen2=3DValidatingChar(:,2);
> TestGen3=3DValidatingChar(:,3);
> % DEFINING THE NETWORK
> % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> S1 =3D 100;%120 =A0 =A0 =A0 =A0 =A0

How did you determine this value for S1?? See my post on pretraining
advice
.
Google Groups

greg-heath pretraining-advice

Sort by date to find the original post.

Also see my posts on how to choose H (=3DS1.)

greg-heath Neq Nw

> net =3D newff(minmax(alphabet),[S1 =A0S2],{'logsig' 'logsig'
> 'logsig'},'traingdx');%traingdx =A0traingdm =A0trainlm traincgf,

Delete one of the logsigs. You only have two layers of weights.
Do you really want the outputs to be restricted to the open
interval (0,1)? If not, use the default 'purelin' for output.

> net.LW{2,1} =3D net.LW{2,1}*0.01;
> net.b{2} =3D net.b{2}*0.01;

Delete the above. initialization is automatic.

> net.performFcn =3D 'sse'; % Sum-Squared Error performance function

Delete and use the default 'mse'.

> net.trainParam.goal =3D 0.10; % Sum-squared error goal.

Use mean(var(targets))/100 for the mse goal.

> net.trainParam.show =3D 10; % Frequency of progress displays
> (in epochs).
> net.trainParam.epochs =3D 5000; % Maximum number of epochs to
> train.

Use the default of 100 with the default trainlm. If insufficient
increase
to  200 or more.

> net.trainParam.mc =3D 0.95;%0.65;% % Momentum constant.
> mc=3D0.65 and s1=3D100 good memorization

Delete and use defaults of trainlm.

> % TRAINING THE NETWORK
> % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> P =3D alphabet;
> T =3D targets;
> [net,tr] =3D train(net,P,T);
>
> % TRAINING THE NETWORK WITH NOISE.


Why are you doing this??

> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> netn =3D net;
> netn.trainParam.goal =3D 0.60; % Mean-squared error goal.

Before you used 'sse'?

> netn.trainParam.epochs =3D 1000;

Can make it much smaller if you use 'trainlm'

> T =3D [targets targets targets targets targets targets];

Wha?

> for pass =3D 1:20
> P =3D [alphabet, alphabet, ...
> (alphabet + randn(Sa,Qa)*0.1), ...
> (alphabet + randn(Sa,Qa)*0.2), alphabet +
> randn(Sa,Qa)*0.3,alphabet];
>
> [netn,trn] =3D train(netn,P,T);
> end

The above makes no sense. You are training the net 20 times
but doing nothing with it the 1st 19 times...?


> % SIMULATION OF THE NETWORK
> % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>
> Y =3D sim(netnn,TestMem); =A0% 1 pe =A0&2 te 3 sheen =A0
> Yy1 =3D sim(netnn,TestGen1); =A0% 1 pe =A0&2 te 3 sheen =A0
> Yy2 =3D sim(netnn,TestGen2); =A0% 1 pe =A0&2 te 3 sheen =A0
> Yy3 =3D sim(netnn,TestGen3); =A0% 1 pe =A0&2 te 3 sheen =A0
> disp([round(Y) round(Yy1) round(Yy2) =A0round(Yy3)]);
>
> first im training on ideal data then on some noisy data and
> then check the results by applying the characters on which i
> have trained the network and on some new data but it give
> good results on data on which i have trained but wrong
> results on new data....simple to say it is 100% good in
> memorization but 100% wrong in generalization...how to
> improve.....thanks in advance....zaheer ahmad

Partition your data into separate training, validation and test
subsets. The training set is used to estimate weights. The
error on validation set is used to determine network (e.g., S1)
and training algorithm paraneters 9e.g., No. of epochs).
The generalization error is estimated via the error on the test set.

If you want to see how robust the net is you can

1. add random noise to the weights and plot test set error
vs weight noise level
2. add random noise to the test set inputs and plot test set error
vs noise level using the original weights.

Hope this helps.

Greg