Path: news.mathworks.com!not-for-mail
From: "zaheer ahmad" <ahmad.zaheer@yah00000.com>
Newsgroups: comp.soft-sys.matlab
Subject: MLP Optimization Problem ( generalization problem ) -- on OCR data--Urdu
Date: Fri, 28 Nov 2008 17:45:04 +0000 (UTC)
Organization: IMS
Lines: 102
Message-ID: <ggpan0$s2d$1@fred.mathworks.com>
Reply-To: "zaheer ahmad" <ahmad.zaheer@yah00000.com>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1227894304 28749 172.30.248.37 (28 Nov 2008 17:45:04 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Fri, 28 Nov 2008 17:45:04 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 706613
Xref: news.mathworks.com comp.soft-sys.matlab:503676


Dear All

I am developing an OCR (Urdu) but having 'goal doesnt meet' problem.

my network is

Input =400 also reduced and checked on 100 and 144
output=54
Hidden layer = 20 but checked on 30,40,50,60,70,80,90,100 upto 250

Sample size i tried to train th net are 
5400   (i.e. 100*54=5400) but also checked on 
540    (i.e. 10*54=540) and 
1080   (i.e. 20*54=1080) and
1350   (i.e. 25*54=1350) and
2700   (i.e. 50*54=2700)
where 54 are the number of character and 100,10,20,25 and 50 are samples of each character. 

i tried on using traingdx, trainlm and trainscg( because of out of memory error ) with both   mse and sse.

i dont know why it doesnt reach to the gaol the goal=0.1 for traingdx (or goal= 0.009 for tranlm)
some time it reach to goal but doesnt recognise test data.

the code is given as below:


clear;clc;

% SET CHARACTERS: 
Alphabet =Alpha4Train;%Alphabet =Alphabet(:,1:100);
Target=TargetSet;%Target=Target(1:100);
[S1,Qa] = size(Alphabet);
[S2,Q] =size(Target);

% DEFINING THE NETWORK
% ====================
H1 =120 ;%115=10  120=0 with mc=0.5   120=2...80....200=met for 10 char ...150     120  for 10 alphas        

net = newff(minmax(Alphabet),[H1 S2],{'logsig' 'logsig'},'traingdx');%trainrp  trainscg
%%%%traingdx  traingdm  trainlm traincgf,   net = newff(minmax(alphabet),[S1 S2],{'logsig' 'logsig'},'traingdx');

net.performFcn = 'sse'; % sse   Sum-Squared Error performance function
net.trainParam.goal =0.10;% mean(var(Target))/100; %0.10;% 0.009;% Sum-squared error goal.
net.trainParam.show = 10; % Frequency of progress displays (in epochs).
net.trainParam.epochs = 95000; %5000 Maximum number of epochs to train.
%  net.trainParam.mc = 0.95;%0.65;% % Momentum constant.  mc=0.65 and s1=100 good memorization
%  net.trainParam.mem_reduc =99999;
% net.trainParam.lr=0.01;%Learning rate
% net.trainParam.lr_inc=1.9;
% net.trainParam.lr_dec = 0.5;

% TRAINING THE NETWORK 
% ====================

P = [Alphabet,Alphabet,Alphabet,Alphabet,Alphabet,Alphabet,Alphabet];
T = [Target,Target,Target,Target,Target,Target,Target];

[net,tr] = train(net,P,T);

% TRAINING THE NETWORK WITH NOISE...GET DIRTY FOR GOOD RESULTS AT THE END
% =======================================================================
netn = net;
netn.trainParam.goal =0.01;% mean(var(Target))/100; %0.009;%mean(var(Target))/100; % Mean-squared error goal.
netn.trainParam.epochs = 85000;%500
netn.trainParam.show = 10; %%% Frequency of progress displays (in epochs).

T = [Target,Target,Target,Target,Target,Target,Target];
P = [(Alphabet + randn(S1,Qa)*0.2), Alphabet + randn(S1,Qa)*0.3, Alphabet + randn(S1,Qa)*0.3,Alphabet,Alphabet,

(Alphabet + randn(S1,Qa)*0.2), Alphabet + randn(S1,Qa)*0.3];
[netn,trn] = train(netn,Alphabet,Target);

%   load netxxx1010; ImProc(netn,net);

save netgdx2115;


%%%%%%%%%%
i have only 100 samples for each character to i have used 

P = [Alphabet,Alphabet,Alphabet,Alphabet,Alphabet,Alphabet,Alphabet];

to the get inequality 

Neq >~ r* Nw       where (~2 < r < ~ 64).    as described by Greg Heath in posts.
so it doesnt need to tell that i have tried to follow Greg Heath rule rule for choosing hidden layer.
even tried to overrule it some times but all in vain.

the comments in the code shows the values i have tested, so i have not omitted the comments for yours
reading despite it make the code reading a bit difficult, hope no one will mind.

thanks
zaheer ahmad