Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!k17g2000yqh.googlegroups.com!not-for-mail
From: Greg Heath <heath@alumni.brown.edu>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Pattern Classification using Neural Network ( newff)
Date: Sun, 8 Nov 2009 02:42:39 -0800 (PST)
Organization: http://groups.google.com
Lines: 112
Message-ID: <d97873af-2fd8-4956-9e02-77a8a2aa4419@k17g2000yqh.googlegroups.com>
References: <hco1vf$ndj$1@fred.mathworks.com>
NNTP-Posting-Host: 69.141.163.135
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
X-Trace: posting.google.com 1257676959 13100 127.0.0.1 (8 Nov 2009 10:42:39 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Sun, 8 Nov 2009 10:42:39 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: k17g2000yqh.googlegroups.com; posting-host=69.141.163.135; 
	posting-account=mUealwkAAACvQrLWvunjg50tRAnsNtJR
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 
	2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729),gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:583317


On Nov 2, 8:45 pm, "Kishore " <kishore3...@yahoo.co.in> wrote:
> Hello,
>
> I am trying to classify a 4 class problem (each class has
> 20 features ) using neural network.
> So, in order to reduce the complexity, i used newff function to
> get the weights.
>
> The  problem is i am not very familiar with newff function usage
> ( the samples are not classified properly- same sample set is being
> classified welll using k nearest neighbour and bayesian techniques).
>
> It would be great if i can get feed back on the usage of  this
> newff sequence.
>
> %%%%%%%%
>
> net = newff(training_data',group',no_hiddenLayer);

help newff
doc newff

You have accepted the PURELIN output default. For classification,
LOGSIG is superior. The output then represents the posterior
probability (conditional on the input) for class "1".

> % Create a new feed forward network. 20 neurons in the hidden layer.

H = 20 may be far too many. It's best to use the smallest
satisfactory
value which is usually found by trial and error.

For an I-H-O MLP, Ntrn training vectors, size(p) = [I Ntrn],
size(t) = [ O Ntrn] and training without regularization, a good rule
of thumb is Neq >> Nw where

Neq = Ntrn*O = Ntrn*4 = No. of training equations.
Nw = (I+1)*H+(H+1)*O  = No. of unknown weights


Search on

greg-heath Neq Nw
greg-heath pretraining advice newbies

for details

> %training data is a matrix of training samles.
> %group is a matrix, where each row is for example
> [1 0 0 0] or [0 1 0 0] or [0 0 1 %0] or [0 0 0 1]
> based on the class to which the  training sample belongs.

Each column should be an output target.

size(t) = [ 4 Ntrn]

I don't recommend using the transpose operator in
the call of newff.

> %so basically i have 4 outputs.
>
> % training parameters -i took this from an example
> net.trainParam.goal = 0.1;

Looks OK:

You would get lower than this if the correct class
output is 0.7 and the other outputs are 0.3

> net.trainParam.show = 20;
> net.trainParam.epochs = 40;
> net.trainParam.mc = 0.95;

delete the last 2 and settle for the defaults in TRAINLM.

help trainlm
doc trainlm

> [net,tr] = train(net,training_data',group');


> W = net.IW{1};
> V = net.LW{2};

Incorrect see the documentation

> W =W';
> V = V';
>
> the best weights are obtained.

It is not necessary to explicitly extract the weights
(and the very important thresholds in net.b).
>
> Now i do the testing...and classify based on the 4 outputs
> ( which ever is maximum).
>
> Is this approach correct or am i missing something?

For any input x with correct outputs y0

size(x)  = [ I N]
size(y0) = [ O N]
y        = sim(net,x);
error    = y0-y;
MSE      = mse(error)


Hope this helps.

Greg