File Exchange

image thumbnail

Multi Class SVM

version 1.0 (1.5 KB) by

Model and classify training/test data sets into more than 2 classes with SVM.

3.63636
27 Ratings

287 Downloads

Updated

View License

Support Vector Machines only classify data into two classes. This function removes that restriction by "searching" for the correct class for each row in the test data set.

This code is a clarification and optimization of Anand Mishra's code found here:
http://www.mathworks.com/matlabcentral/fileexchange/33170-multi-class-support-vector-machine

Use only with more than 2 classes, otherwise use svmtrain() directly.

Usage Example:

%% SVM Multiclass Example
% SVM is inherently one vs one classification.
% This is an example of how to implement multiclassification using the
% one vs all approach.
TrainingSet=[ 1 10;2 20;3 30;4 40;5 50;6 66;3 30;4.1 42];
TestSet=[3 34; 1 14; 2.2 25; 6.2 63];
GroupTrain=[1;1;2;2;3;3;2;2];
results = multisvm(TrainingSet, GroupTrain, TestSet);
disp('multi class problem');
disp(results);

Comments and Ratings (69)

can anyone please tell me that , how to use this svm in fingerprint verification?

SUNNY NEGI

Hello everyone, Can anybody tell me how to resolve this problem....
Error using seqminopt>seqminoptImpl (line 198)
No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)
[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 499)
    [alpha, bias] = seqminopt(training, groupIndex, ...

Error in multisvm (line 20)
    models(k) = svmtrain(TrainingSet,G1vAll);

Error in svmfinal (line 56)
Class = multisvm(TRAINING,Y,X2);
 

Ambika PS

Emna Krichene

I tried the code, and the problem is that it gives in all cases 1 or 2 or 3 as classification, however it must only predict 0 or 1 or -1 !

I can't see where is the problem !

any help please?

veera V

TrainingSet2=[5.65 20.3 14.7;
        5.65 20.3 0.2646;
        0.341 0.341 0.341
    0.07 0.07 0.0116;
        0.0233 0.4163 0.1731;
        0.072 0.072 0.3708;];
    GroupTrain2=[1;2;3;4;5;6;];
    TestSet2=[
       0.072 0.072 0.3708];
This is my training and i am getting result as 3 it should be 6 actually

Lucas Kreiß

I have to agree with Alessiox (23 Jan 2016):

The prediction score for a correct classification amongst several classes is not tested.

Furthermore the user has no access to the scores, AUC values etc.

This code can classify and might be useful for rough estimation. But it cannot help with analysing how good your training and test data actually is and how accurate the classification is done in each case.

Goma Najah

@Abdul Latif

راسلني على:

goma.najah@yahoo.com

Abdul Latif

Hi,
i have problem with my code. I want to make three class or groups in svm, but this code don't work.
Can anybody help me please?
Thank you very much.

Code :
GLCM2 = graycomatrix(I3,'Offset',[4 4]);
    stats = graycoprops(GLCM2,{'contrast','homogeneity','energy','Correlation'})
        
    a = stats.Contrast;
    b = stats.Correlation;
    c = stats.Energy;
    d = stats.Homogeneity;
    
    set(handles.edit1, 'String',a);
    set(handles.edit2, 'String',b);
    set(handles.edit3, 'String',c);
    set(handles.edit4, 'String',d);

    load class_uji2.txt;// document text class or group with three class {0,1,2}
    load data_uji2.txt;// document training data
    uji1=data_uji2(:,1:2:3:4);
    massa = svmtrain(uji1,class_uji2)
        
    uji = svmclassify(massa, [a b c d]) // [a b c d] test data with glcm
    if uji == 0
        hasil = 'Normal'
    else if uji == 1
hasil = 'Benign Cancer'
else
        hasil = 'Malignant Cancer'
    end
end
    
    set(handles.edit5, 'String',hasil);

hi cody
I use this code for a 3 class classification problem. how to get accuracy of the result using K-fold cross validation? please help me.
thank you very much.

Dylan Liu

Hi
I tried this code, it is working fine on given examples, but when I tried to solve my classification problem, following error appears

Error using seqminopt>seqminoptImpl (line 198)
No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)
[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 499)
    [alpha, bias] = seqminopt(training, groupIndex, ...

Error in multisvm (line 20)
    models(k) = svmtrain(TrainingSet,G1vAll);

Sneha Jain

hii...i am doing retrieval of images..i have classified my target image using code of your code...there are three classes namely, 1,2 and 3..the class of my target image is displayed as 1. now i want to retrieve all images of class 1 and display the images. how to do that??

how to use same code on excel sheet data???? plz tell

Anand Mishra

Anand Mishra (view profile)

Hi Cody,
One user Can Liu asked for some error in codes for his code.
The code is
R=[1 2 3 4;1 2 3 4;3 4 5 6;7 8 9 10;7 8 9 10]
C=[2 2 1 3 3]'
T=[1 2 3 4;7 8 9 10;3 4 5 6]
[IT1]=multisvm(R,C',T)
After running this code by my file result shows correctly but your code shows incorrect result as
 IT1 =

     2
     3
     3

It should be
IT1 =

     2
     3
     1

I think there is some error in your code....or anything else?

Manish Sahu

A good code & I classified 3 classes using this version. But I am not getting the boundary lines for the classified data.

Alessiox

The "result" heuristic isn't that great. In this case you pick the "first point" and then the break kicks in. That is not correct: in a one-vs-all scenario you must take into account the prediction score for a correct classification amongst several classes

Ok your code running well, its error because the name of the function is not set correcly,thank you

I still get an error code for the group
Error using multisvm (line 11)
Not enough input arguments.
Please explain how we solve that problems
Thank you

hi
I use this code for a 4 class classification problem. even when i use the training set to test algorithm, I only get two classes could you help me?

I got 3 classifications using multiclass svm. But unable to get the kernel function. How we ca change the kernel function to drw the fugure?

Manu BN

Manu BN (view profile)

Plz note that the group must strictly be a row vector. For e.g to classify into three classes: (where each class is of 10*13)

T = [Class1;Class2;Class3];
C = [ zeros(1,10) ones(1,10) 2*ones(1,10) ];

The dimensions of T is 30*13
The dimensions of C is 1*30

Guess u got the point !!!

Hi,

I have a problem with this code. I try it with a new example make by myself but it isn't working as it should.

As you can see, I train with 6 samples (groups) orderer 1 by 6 and the test is the same.

When I try this code, the first three are correct, but the last three are wrong.

Can anybody help me please?

CODE:

train=[1 10 100; 2 20 200; 3 30 300; 4 40 400; 5 50 500; 6 60 600];

group = [1;2;3;4;5;6];

test = [0.2 2 20; 2.1 21 210; 3.2 32 320; 4.1 41 410; 5.1 51 510; 6.1 61 610]

multisvm(train, group, test)

A B

A B (view profile)

what do you do to run correctely this code

hai,

     i am working on query classification using matlab 2013b. i planned to using svm. but there is no svm tool box in matlab 2013b. can u help to add the svm tool for query classification.

He has worked very well, thank you very much

As I can reference the code owner?

can you help me with the reference to call it on my thesis, correctly?

Thanks!

Jack

Jack (view profile)

Thank you for this effective code. But, can you please add an evaluation code for this classification, and how we can get the classification accuracy for that. ( on the same example please) .

Thanks in advance.

Dear Cody..

i just wanna ask,
how do we modify the holdout sampling coding to be the bootstrap sampling coding. The simplest coding.

Deepa AS

I loaded the training set and my test data set.But it always throws an error saying undefined function or variable GroupTrain.Can anybody please tell me how to overcome this error?

sunil

sunil (view profile)

Dear Cody, please help me in developing SVM to recognize Handwritten digits from 0 to 9.I tried using multiclass SVM function, but of no use.I'm waiting for your response. In the above program, how to consider values for TrainingSet,TestSet and Group Train. I Have ICDAR13 dataset ,over which i calculated statistical features for over 30 samples for each digits(0 -9).from next i strucked.please give suggestion.

Samuel Pedro

Amruta

Amruta (view profile)

is it GroupTrain vector must be column vector or row vector?
while i am running it for classfication in one program using 'group' as column vector giving me correct answer but for another program it is giving me answer as 1 for all.Then i was tried it for 'group' as row vector it is giving me 6 correct answers out of 10 classes.
Then what to do to get correct classification output?

what is your output?? Everytime it is showing 1.. am i not get which group it is belong?? for example your test set value 3 34 is a group of 2 but it is also showing 1 , why?

Dear All;

I am trying to use this function for more than three classes , it does not work , in the group train , I have { 1,2,3,4}
could you help me

Anwesh

Anwesh (view profile)

Amazing code...I have got the results...how can i give cell aray strings as grouptrain input

Shailesh

Hello, I have implemented the code, but getting output as all one only. please help me as soon as possible..

@aabiya Group train is the class of the training set.

aabiya

aabiya (view profile)

please let me know how to calculate the group train matrix. I have calculated the train and the test set but can't understand group train variable.

Amarjot

Amarjot (view profile)

Thanks for the code!

Peter

Peter (view profile)

Thanks for the code. With the latest version of MATLAB (2014a) you don't need the Bioinformatics Toolbox to run the code.

ashok

ashok (view profile)

sir, i think the code will not check for the next class predicted, please check if it is 5 class

Borja Heriz

Thank you so much Cody. It has been really helpfull for my current project!!!!!

Please cody help me how to put matrix of my image data set in training set,test set and group set ?II am new so no idea!

I do not undersand how to do these steps for my data set
TrainingSet=[ 1 10;2 20;3 30;4 40;5 50;6 66;3 30;4.1 42];
TestSet=[3 34; 1 14; 2.2 25; 6.2 63];
GroupTrain=[1;1;2;2;3;3;2;2];

Hamid Ramezani

Hamid Ramezani (view profile)

It seems that the code considered each set of data individually, and not jointly. Do you have any reference for your algorithm? If so, I would be thankful if you share that.

Walid

Walid (view profile)

Hi, and thanks for the code. Although I think there is a minor mistake in the code. In the last line, before the "end", the line "result(j) = k;" should be "result(j) = u(k);".

Javed

Javed (view profile)

mahdi

mahdi (view profile)

Hi.Thanks for your implementation but I think it is incorrect. When you return 0 or 1 from svmclassify, you maybe have several zeros or ones with different classes!!!
I think you need to compute the real values(before execute sign function) and assign the test data to the class which has the biggest positive ouput.

Lulin

Lulin (view profile)

This code can be improved if it can determine a class membership when a test observation belongs to multiple classes.

raji

raji (view profile)

thanks for the updated contents!!! very much helpful...

joy barbosa

thank you so much for sharing your code sir Cody. It saved my day!!! 2thumbs up!

joy barbosa

B mohan

I am doing project on facial expression recognition using LBP(Local Binary pattern). i.e finding expressions whetherit is fear, anger, surprise, disguist, happy, joy. First I loaded jaffee database. Then applied preprocessing on it. And the features are extracted using LBP(Local Binary pattern). I don't know how to use multiclass SVM(Support vector Machine). Can this code be useful to my project. What shoud I take input TrainingSet ,GroupTrain,TestSet. Please help me.

B mohan

I am doing project on facial expression recognition using LBP(Local Binary pattern). i.e finding expressions whetherit is fear, anger, surprise, disguist, happy, joy. First I loaded jaffee database. Then applied preprocessing on it. And the features are extracted using LBP(Local Binary pattern). I don't know how to use multiclass SVM(Support vector Machine). Can this code be useful to my project. What shoud I take input TrainingSet ,GroupTrain,TestSet. Please help me.

Nimmi

Nimmi (view profile)

Can this be used in model predictive control algorithm.

Subha

Subha (view profile)

Sir is it possible for us to add weights for the feature with this code??? if so how to proceed with it..?

Subha

Subha (view profile)

Thank you sir.. now trying out with best possible combination...

Cody

Cody (view profile)

  • 1 file
  • 287 downloads
  • 3.63636

If you are getting all 1s, you should try using different options for the svmtrain function. for example:
options = statset('MaxIter',2000000);
models(k) = svmtrain(X_Train,G1vAll,'options',options,...
        'kernel_function','rbf','rbf_sigma',16);

Subha

Subha (view profile)

sir,
is it possible to plot confusion matrix for this program... if so how to do tat.. we have tried many combinations but does not working...
....
else any method to calculate the performance of the overall system from test data's..

Subha

Subha (view profile)

oh.. ok sir.. now its working.. thank a lot...

Cody

Cody (view profile)

  • 1 file
  • 287 downloads
  • 3.63636

Ok, you are not using the function correctly.

load your .mat file, then pass your new variables as parameters to the function.

Subha

Subha (view profile)

Sir,
I've loaded data and worked in 2 class problem. its works well...
this is my code.. i've just loaded the data and didnt change anything more.
....
function [result] = multisvm(TrainingSet,GroupTrain,TestSet)
load TrainingSet
load GroupTrain
load TestSet
u=unique(GroupTrain);
numClasses=length(u);
result = zeros(length(TestSet(:,1)),1);

%build models
for k=1:numClasses
    %Vectorized statement that binarizes Group
    %where 1 is the current class and 0 is all other classes
    G1vAll=(GroupTrain==u(k))
    models(k) = svmtrain(TrainingSet,G1vAll,'kernel_function','rbf');
end

%classify test cases
for j=1:size(TestSet,1)
    for k=1:numClasses
        if(svmclassify(models(k),TestSet(j,:)))
            break;
        end
    end
    result(j) = k;
end

where,
training set is 16*4
test set is 4*4
grouptrain is 16*1
I'm using Matlab (R2010a)

Cody

Cody (view profile)

  • 1 file
  • 287 downloads
  • 3.63636

It sounds like you need to learn how to load .mat files. Simply type 'load xxx.mat' where xxx is the file name. You would then need to sample your data into training, test, and label sets for use with this function.

Can you show the code you are using?

Subha

Subha (view profile)

Hello Sir,
Thanks for the code.. it works well when we give data... but i'm working with large set of data, where i need to load it(.mat)..
I cant load data, if i do so then it showing error message
??? Undefined function or variable 'GroupTrain'.

Error in ==> neuro_multisuport at 40
u=unique(GroupTrain);
where grouptrain is a column vector(16*1) in .mat format
.. can u please suggest me how to modify the code when i load data...

Niyalatul

Hello,
Thanks for your program. I am still new to learn this topic. This is very helpful

Niyalatul

Niyalatul

MATLAB Release
MATLAB 7.14 (R2012a)
Acknowledgements

Inspired by: Multi Class Support Vector Machine

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video