|
On Dec 7, 5:56 pm, Bita <b.mirshafie...@gmail.com> wrote:
> On Dec 5, 3:51 am, Greg Heath <he...@alumni.brown.edu> wrote:
> > On Dec 3, 9:14 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> > > On Dec 3, 6:20 pm, Bita <b.mirshafie...@gmail.com> wrote:
> > > > On Dec 3, 9:59 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> > > > > On Dec 2, 2:47 pm, Bita <b.mirshafie...@gmail.com> wrote:
>
> > > > > > I'm working on my project which is about face recognition, I used
> > > > > > fisherface for feature extraction part, and I should use rbf neural
> > > > > > network as it's classifier.
-----SNIP
> > > > > > I use MATLAB software and a database of 400 images with 40
> > > > > > people(classes) and 10 image per individual, each image is 112x92.(4
> > > > > > images of each class is test set and 6 images of them is training set,
> > > > > > so I have 240 train images).
> > > > > > There are 39 features for each image extracted(so there are 39 nodes
> > > > > > in input layer) and I use KMEANS command to cluster training images
> > > > > > and clustered them into 7 clusters( according to what I've read in a
> > > > > > book that the cluster numbers should be like this : (clusternumbers)^3
> > > > > > = training images numbers)
>
> > > > > PHOOEY! Have you obtained a colorcoded 2-D visualization of the data?
>
> > > > > >and at last there are 40 nodes in hidden
> > > > > > layer in which the outputs are between 0-1 and the image that is
> > > > > > nearest to 1 is belonging to that class.
>
> > > > > NEWFF with 7 hidden nodes and 40 TANSIG outputs?
>
> > > > > The input is assigned to the class that has the highest output.
>
> > > > > > The rbf part doesn't work. Is there any problem with my network?
>
> > > > > How are we supposed to know? What NNToolbox programs are you
> > > > > using? Where are some relevant samples of code?
>
> > > > > > could it be because of low number of training images?
>
> > > > > Probably not.
>
> > > > > size(p) = [I Ntrn] = [39 240] % input
> > > > > size(t) = [O Ntrn] = [40 240] % Output
> > > > > Neq = Ntrn*O = 9600 % No. training equations
> > > > > Nw = (I+1)*H + (H+1)*O = O +(I+O+1)*H = 40+80*H % No. unknown
> > > > > weights
>
> > > > > Neq >> Nw % Robust weight estimation
>
> > > > > H << (Neq-O)/(I+O+1) = 956/8 ~ 119 % Hidden Nodes
>
> > > > > Please post relevant code.
>
> > > > > P.S. If you are not using the NNToolbox, just form 1 cluster per class
> > > > > by averaging the training data. Assign inputs to the nearest cluster
> > > > > center. Iteratively create new clusters from misclassified training
> > > > > data.
>
> > > > > Hope this helps.
>
> > > > > Greg
>
> > > > I don't use NNToolbox, and for clustering I suppose to cluster my data
> > > > by KMEANS and then apply gaussian function:
>
> > > > Exp(-(norm(x(i)-cm)^2)/2*norm(variance))
>
> > > > I think it's different from what you've said, to form clusters just by
> > > > averaging the training data,what's the difference between these two
> > > > ways of making clusters?
>
> > > KMEANS is UNSUPERVISED clustering which ignores class membership.
> > > Useful for determining the characteristics of UNLABELED data. For
> > > classification it tends to be inferior to using SUPERVISED clustering
> > > where class labels are taken into account.
>
> > > > I won't get a good result by using KMEANS clustering?( actually I still haven't )
>
> > > I wouldn't count on it unless you initialized it with the 40 class
> > > centroids.
>
> > > > Should I put my previous code and method away and start anew?
>
> > > First try kmeans initialized with the 40 centroids.
>
> > > > By averaging the training data for each class we will obtain the
> > > > centers for clusters?
>
> > > Yes. Compare with initializing kmeans with the 40 centroids.
>
> > > > > Iteratively create new clusters from misclassified training data.
>
> > > > What does it mean?did you mean that after finding inputs that are the
> > > > nearest to each cluster, average those to find the new centers?
>
> > > Yes. There are several methods
>
> > > 1. BATCH UPDATING:
> > > a. Pass all of the data through; Keep track of all inputs that are
> > > correctly classified.
> > > b. Update centroid of each cluster with the corresponding
> > > correctly classified inputs
> > > c. Create a new cluster with the incorrectly classified input that
> > > is the farthest from it's nearest same class centroid.
>
> > > 2. SEQUENTIAL UPDATING
>
> > a and b are optional.
>
> > > a. Reorder the data so that each group of 40 inputs contains one
> > > input per
> > > class.
> > > b. Randomly reorder the positions of the inputs within each group
> > > of 40.
>
> > > Loop over inputs:
> > > c. Pass a single input through.
>
> > d. If correctly classified update the nearest same class
> > centroid
> > e. If incorrectly classified, use it to create a new cluster
> > centroid.
> > f. Go back to 2c.
>
> > > 3. ALTERNATING BS UPDATING
> > > a. Batch update for a single epoch
> > > b. Sequentially update for a single epoch
> > > 4. ALTERNATING SB UPDATING
> > > a. Sequentially update for a single epoch
> > > b. Batch update for a single epoch
>
> > > > Thanks Greg for answering, and sorry for all these questions cause
> > > > I've got a lot of questions and really confused.
>
> > > That is why it is better to first learn how to use the existing NNTB
> > > functions.
>
> > Hope this helps.
>
> > Greg
>
> First, Thanks a lot for your comprehensive answer.
>
> But if you see in my previous post I mentioned the gaussian function.
> I should find the clusters as you explaind and then use them in that
> function to obtain the value of each hidden node?
Yes
> My the other question is that you introduced several methods for
> updating, but which of them is better?
The best method depends on the data and how each individual programmer
handles details. Beginners should start with the easiest (BATCH)..
>I found the first method(batch
> updating) more simpler than others, but what do you suggest me to
> implement and put time on it for this neural network?
BATCH sounds good.
Let's start from the beginning.
1. You are designing a classifier; not a function estimator for
regression. Therefore, I don't recommend starting with k-means
obtained by
unsupervised learning of the unlabeled class mixture.
2. Start with one cluster for each class and use the class centroid =
mean(ptrni,2) (i=1:40) and average mixture (NOT CLASS) variance =
mean(var(ptrn')) to create Gaussian (help RADBAS, doc RADBAS) hidden
nodes.
3. Use a linear (PURELIN) output activation function and obtain the
weights
using backslash.
4. Assign the input to the class corresponding to the maximum output.
etc.
Hope this helps.
Greg
|
Feed for this Thread