Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!t2g2000yqm.googlegroups.com!not-for-mail
From: Greg Heath <heath@alumni.brown.edu>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Neural Networks Cant Generalize..Results error in New Data
Date: Tue, 2 Dec 2008 14:06:26 -0800 (PST)
Organization: http://groups.google.com
Lines: 98
Message-ID: <94a8e012-1f62-4a32-aa30-5268928dbbc6@t2g2000yqm.googlegroups.com>
References: <g9ole2$ins$1@fred.mathworks.com> <8436a506-ab26-4400-b349-3e2649bd757c@26g2000hsk.googlegroups.com> 
	<gaaq75$1ro$1@fred.mathworks.com> <gfv3h6$j7p$1@fred.mathworks.com>
NNTP-Posting-Host: 68.39.98.10
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Trace: posting.google.com 1228255586 21030 127.0.0.1 (2 Dec 2008 22:06:26 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Tue, 2 Dec 2008 22:06:26 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: t2g2000yqm.googlegroups.com; posting-host=68.39.98.10; 
	posting-account=mUealwkAAACvQrLWvunjg50tRAnsNtJR
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 
	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; 
	.NET CLR 2.0.50727; .NET CLR 3.0.04506.30; Seekmo 10.0.341.0),gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:504513


On Nov 18, 2:03 pm, "zaheer ahmad" <ahmad.zah...@yah00000.com> wrote:
> Dear Greg Heath thanks for your reply and sorry for my late response, i was busy in some other projects and didnt add this thread to my watchlist( i think)...
> i implemented all your suggestions in my code and tested the code with various angles, but unfortunately i am having same problem.
> 1.  removed one logsig ( no doubt it was not required)
> 2. brackets () after function name is not required but matlab not warn or give error if written ( i used it by mistake....other languages habit...but now removed )
> 3. Number of Hidden layers were find out through hit and trial exercise.
> 4. i had already checked with ' sse '  and traingdm  trainlm traincgf but in vain.
> 5. i was looping ( 20 times ) just to get good result using increased random values( but i agree found unnecessary )
> 6. from new data i meant test data ( data on which net is not trained but similar to trained data)
>
> My Project is about Urdu OCR...you might know about Urdu its national language of Pakistan..there is no lower case alphabets(atleast not like roman) in it. so be sure i am not trying to match up a,b against  A,B....
>
> As stated earlier i have checked/implemented all your suggestions..but i think the problem is not in the training process/code....where? i am not sure....but i have the details of my project so sharing with you:
> i am having 54 classes / patterns/alphabets on which trying to train the net....i have  trained the net once on  10 samples of each alphabet, checked results but not good, the same way trained the net on 25, samples and then on 50 samples of each pattern i.e.  a total of 540,1350,2700 samples . but the results are not encouraging (or say disappointing) and gives same result no matter how many samples are used. note that it converges on number of hidden layer=120 and also on =100.
> The main problem i am facing is...the trained net can recognise/match alphabets on which it was trained but can not match similar data on which it was not trained on.

Maybe your classes are not well defined
and have to be partitioned into subclasses
via clustering (e.g., k-means).

Overlay the plot of each misclassified character
(blue) on the plot of the mean of the class to
which they were assigned (red) and the plot of
the mean of the correct class (black)clustering.

This should give some insight into the difficulty.

> code with amendments is as below:
>
> clear;clc;
>
> % SET CHARACTERS:
> alphabet =Alpha4Train;
> targets=TargetSet;
> [Sa,Qa] = size(alphabet);
> [S2,Q] =size(targets);

Well, what are they?

> %%% Q=Qa
> % DEFINING THE NETWORK
> % ====================
> S1 =120 ;

How did you determine this?

> net =newff(minmax(alphabet),[S1 S2],{'logsig' 'logsig'},'traingdx');

Why not standardize the inputs and use
tansig in the hidden layer?

> % i have checked for trainlm toooo
>
> net.performFcn = 'sse'; % checked on mse tooo

Forget sse

> net.trainParam.goal = 0.10;%checked on mean(var(targets))/100;

Looks too high. Probably something wrong.

> net.trainParam.show = 20; % Frequency of progress displays (in epochs).
> net.trainParam.epochs = 15000; %5000 Maximum number of epochs to train.
> net.trainParam.mc = 0.5;%checked on 0.65 and 0.95 too
>
> % TRAINING THE NETWORK
>
> P = alphabet;
> T = targets;
>
> [net,tr] = train(net,P,T);

Where is your error calculation?

> % TRAINING THE NETWORK WITH NOISE...GET DIRTY FOR GOOD RESULTS AT THE END
> netn = net;
> netn.trainParam.goal = 0.60;
> netn.trainParam.epochs = 10000;%500
> netn.trainParam.show = 20; %%% Frequency of progress displays (in epochs).
> T = [targets targets targets];
> P = [(alphabet + randn(Sa,Qa)*0.2), alphabet + randn(Sa,Qa)*0.3,alphabet];

The noise has to be scaled to the standard deviation
of the signal. Have you actually looked at these noisy
characters?

Q ~= Qa.

> [netn,trn] = train(netn,P,T);
>
> I think i am having problem not in code and training algos but something else.

Concentrate on the noiseless characters.

Hope this helps.

Greg