version 1.0.0.0 (1.5 KB) by
Cody

Model and classify training/test data sets into more than 2 classes with SVM.

Support Vector Machines only classify data into two classes. This function removes that restriction by "searching" for the correct class for each row in the test data set.

This code is a clarification and optimization of Anand Mishra's code found here:

http://www.mathworks.com/matlabcentral/fileexchange/33170-multi-class-support-vector-machine

Use only with more than 2 classes, otherwise use svmtrain() directly.

Usage Example:

%% SVM Multiclass Example

% SVM is inherently one vs one classification.

% This is an example of how to implement multiclassification using the

% one vs all approach.

TrainingSet=[ 1 10;2 20;3 30;4 40;5 50;6 66;3 30;4.1 42];

TestSet=[3 34; 1 14; 2.2 25; 6.2 63];

GroupTrain=[1;1;2;2;3;3;2;2];

results = multisvm(TrainingSet, GroupTrain, TestSet);

disp('multi class problem');

disp(results);

Cody (2021). Multi Class SVM (https://www.mathworks.com/matlabcentral/fileexchange/39352-multi-class-svm), MATLAB Central File Exchange. Retrieved .

Created with
R2012a

Compatible with any release

**Inspired by:**
Multi Class Support Vector Machine

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!Create scripts with code, output, and formatted text in a single executable document.

akca -It's running but results are suspecious!

Roy LimHi, May i know how to calculate the accuracy ?

mirna watiSinnathambi MurugiahFan YangI have to say your code is much better than the code you refer to.

thomas lassError in multisvm (line 18)

models(k) = fitcsvm(TrainingSet,G1vAll);

Error in multiclasssvmex (line 8)

results = multisvm(TrainingSet, GroupTrain, TestSet);

I ran the codes on matlab 2018a and it is showing the following errors.Please I need the correct codes

hello_world@Cody In your code, you have the following:

models(k) = svmtrain(TrainingSet,G1vAll);

How to use this code with Libsvm precomputed kernel? In particular, how to use it here:

model = svmtrain(TrainLabels, double(K), '-t 4');

Note that here 'svmtrain' is a Libsvm function not to be confused with MATLAB's 'svmtrain' function.

Abhijeet Go-karAkhil Karazhmaaditi bhatejaVishnu PriyaHI!I was able to execute the program after changing the max iterations and kktviolationlevel but the accuracy of the classifier is very less ,I mean I have more number of samples belonging to class 1 and only class 1 is being classified.How do I solve this?

What is the consequence of increasing the kktviolationlevel?

Could someone please answer me?

Vidya BegaHI!I was able to execute the program after changing the max iterations but the accuracy of the classifier is very less ,I mean I have more number of samples belonging to class 1 and only class 1 is being classified.How do I solve this?

Vishnu PriyaHI!I'm getting the same error as SUNNY NEGI

Could someone help me with this problem?

Error using seqminoptIseqminoptImpl (line 198)

No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)

[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 505)

[alpha, bias] = seqminopt(training, groupIndex, ...

Error in multisvm (line 20)

models(k) = svmtrain(TrainingSet,G1vAll);

Error in egsvm (line 24)

class = multisvm(data,label,t_data);

SantosGARUL JOTHI Rthank u

Safa Özernithya nFadzil Hassanhi cody

I use this code for 3 classes problem. how to get accuracy of the result using K-fold cross validation? please help me.

thank you very much.

Sapna Ssunantha rajakumarcan anyone please tell me that , how to use this svm in fingerprint verification?

SUNNY NEGIHello everyone, Can anybody tell me how to resolve this problem....

Error using seqminopt>seqminoptImpl (line 198)

No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)

[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 499)

[alpha, bias] = seqminopt(training, groupIndex, ...

Error in multisvm (line 20)

models(k) = svmtrain(TrainingSet,G1vAll);

Error in svmfinal (line 56)

Class = multisvm(TRAINING,Y,X2);

Ambika PSEmna KricheneI tried the code, and the problem is that it gives in all cases 1 or 2 or 3 as classification, however it must only predict 0 or 1 or -1 !

I can't see where is the problem !

any help please?

veera VTrainingSet2=[5.65 20.3 14.7;

5.65 20.3 0.2646;

0.341 0.341 0.341

0.07 0.07 0.0116;

0.0233 0.4163 0.1731;

0.072 0.072 0.3708;];

GroupTrain2=[1;2;3;4;5;6;];

TestSet2=[

0.072 0.072 0.3708];

This is my training and i am getting result as 3 it should be 6 actually

Lucas KreißI have to agree with Alessiox (23 Jan 2016):

The prediction score for a correct classification amongst several classes is not tested.

Furthermore the user has no access to the scores, AUC values etc.

This code can classify and might be useful for rough estimation. But it cannot help with analysing how good your training and test data actually is and how accurate the classification is done in each case.

Goma Najah@Abdul Latif

راسلني على:

goma.najah@yahoo.com

Abdul LatifHi,

i have problem with my code. I want to make three class or groups in svm, but this code don't work.

Can anybody help me please?

Thank you very much.

Code :

GLCM2 = graycomatrix(I3,'Offset',[4 4]);

stats = graycoprops(GLCM2,{'contrast','homogeneity','energy','Correlation'})

a = stats.Contrast;

b = stats.Correlation;

c = stats.Energy;

d = stats.Homogeneity;

set(handles.edit1, 'String',a);

set(handles.edit2, 'String',b);

set(handles.edit3, 'String',c);

set(handles.edit4, 'String',d);

load class_uji2.txt;// document text class or group with three class {0,1,2}

load data_uji2.txt;// document training data

uji1=data_uji2(:,1:2:3:4);

massa = svmtrain(uji1,class_uji2)

uji = svmclassify(massa, [a b c d]) // [a b c d] test data with glcm

if uji == 0

hasil = 'Normal'

else if uji == 1

hasil = 'Benign Cancer'

else

hasil = 'Malignant Cancer'

end

end

set(handles.edit5, 'String',hasil);

Nani Harniawatihi cody

I use this code for a 3 class classification problem. how to get accuracy of the result using K-fold cross validation? please help me.

thank you very much.

Dylan LiuMaheen JehangirHi

I tried this code, it is working fine on given examples, but when I tried to solve my classification problem, following error appears

Error using seqminopt>seqminoptImpl (line 198)

No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)

[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 499)

[alpha, bias] = seqminopt(training, groupIndex, ...

Error in multisvm (line 20)

models(k) = svmtrain(TrainingSet,G1vAll);

Sneha Jainhii...i am doing retrieval of images..i have classified my target image using code of your code...there are three classes namely, 1,2 and 3..the class of my target image is displayed as 1. now i want to retrieve all images of class 1 and display the images. how to do that??

monica sabharwalhow to use same code on excel sheet data???? plz tell

Anand MishraHi Cody,

One user Can Liu asked for some error in codes for his code.

The code is

R=[1 2 3 4;1 2 3 4;3 4 5 6;7 8 9 10;7 8 9 10]

C=[2 2 1 3 3]'

T=[1 2 3 4;7 8 9 10;3 4 5 6]

[IT1]=multisvm(R,C',T)

After running this code by my file result shows correctly but your code shows incorrect result as

IT1 =

2

3

3

It should be

IT1 =

2

3

1

I think there is some error in your code....or anything else?

Moamen AbdelwahedMarcin GrzywaczManish SahuSasanko Sekhar GantayatA good code & I classified 3 classes using this version. But I am not getting the boundary lines for the classified data.

AlessioXThe "result" heuristic isn't that great. In this case you pick the "first point" and then the break kicks in. That is not correct: in a one-vs-all scenario you must take into account the prediction score for a correct classification amongst several classes

iwan soenandiOk your code running well, its error because the name of the function is not set correcly,thank you

iwan soenandiI still get an error code for the group

Error using multisvm (line 11)

Not enough input arguments.

Please explain how we solve that problems

Thank you

tannaz akbarpourhi

I use this code for a 4 class classification problem. even when i use the training set to test algorithm, I only get two classes could you help me?

Sasanko Sekhar GantayatI got 3 classifications using multiclass svm. But unable to get the kernel function. How we ca change the kernel function to drw the fugure?

Manu BNPlz note that the group must strictly be a row vector. For e.g to classify into three classes: (where each class is of 10*13)

T = [Class1;Class2;Class3];

C = [ zeros(1,10) ones(1,10) 2*ones(1,10) ];

The dimensions of T is 30*13

The dimensions of C is 1*30

Guess u got the point !!!

Mario Subías PérezHi,

I have a problem with this code. I try it with a new example make by myself but it isn't working as it should.

As you can see, I train with 6 samples (groups) orderer 1 by 6 and the test is the same.

When I try this code, the first three are correct, but the last three are wrong.

Can anybody help me please?

CODE:

train=[1 10 100; 2 20 200; 3 30 300; 4 40 400; 5 50 500; 6 60 600];

group = [1;2;3;4;5;6];

test = [0.2 2 20; 2.1 21 210; 3.2 32 320; 4.1 41 410; 5.1 51 510; 6.1 61 610]

multisvm(train, group, test)

A Bwhat do you do to run correctely this code

Srinaganya Gopalrathnamhai,

i am working on query classification using matlab 2013b. i planned to using svm. but there is no svm tool box in matlab 2013b. can u help to add the svm tool for query classification.

Andres SaavedraHe has worked very well, thank you very much

As I can reference the code owner?

can you help me with the reference to call it on my thesis, correctly?

Thanks!

JackThank you for this effective code. But, can you please add an evaluation code for this classification, and how we can get the classification accuracy for that. ( on the same example please) .

Thanks in advance.

aqilah chesulaimanDear Cody..

i just wanna ask,

how do we modify the holdout sampling coding to be the bootstrap sampling coding. The simplest coding.

Deepa ASI loaded the training set and my test data set.But it always throws an error saying undefined function or variable GroupTrain.Can anybody please tell me how to overcome this error?

sunilDear Cody, please help me in developing SVM to recognize Handwritten digits from 0 to 9.I tried using multiclass SVM function, but of no use.I'm waiting for your response. In the above program, how to consider values for TrainingSet,TestSet and Group Train. I Have ICDAR13 dataset ,over which i calculated statistical features for over 30 samples for each digits(0 -9).from next i strucked.please give suggestion.

32sthideAmrutais it GroupTrain vector must be column vector or row vector?

while i am running it for classfication in one program using 'group' as column vector giving me correct answer but for another program it is giving me answer as 1 for all.Then i was tried it for 'group' as row vector it is giving me 6 correct answers out of 10 classes.

Then what to do to get correct classification output?

shayla sharminwhat is your output?? Everytime it is showing 1.. am i not get which group it is belong?? for example your test set value 3 34 is a group of 2 but it is also showing 1 , why?

mustafa alnasserDear All;

I am trying to use this function for more than three classes , it does not work , in the group train , I have { 1,2,3,4}

could you help me

AnweshAmazing code...I have got the results...how can i give cell aray strings as grouptrain input

ShaileshHello, I have implemented the code, but getting output as all one only. please help me as soon as possible..

Abdullah Masroor Hashmi@aabiya Group train is the class of the training set.

aabiyaplease let me know how to calculate the group train matrix. I have calculated the train and the test set but can't understand group train variable.

AmarjotThanks for the code!

PeterThanks for the code. With the latest version of MATLAB (2014a) you don't need the Bioinformatics Toolbox to run the code.

ashoksir, i think the code will not check for the next class predicted, please check if it is 5 class

Borja HerizThank you so much Cody. It has been really helpfull for my current project!!!!!

Sheema KhattakPlease cody help me how to put matrix of my image data set in training set,test set and group set ?II am new so no idea!

I do not undersand how to do these steps for my data set

TrainingSet=[ 1 10;2 20;3 30;4 40;5 50;6 66;3 30;4.1 42];

TestSet=[3 34; 1 14; 2.2 25; 6.2 63];

GroupTrain=[1;1;2;2;3;3;2;2];

Hamid RamezaniIt seems that the code considered each set of data individually, and not jointly. Do you have any reference for your algorithm? If so, I would be thankful if you share that.

WalidHi, and thanks for the code. Although I think there is a minor mistake in the code. In the last line, before the "end", the line "result(j) = k;" should be "result(j) = u(k);".

JavedmahdiHi.Thanks for your implementation but I think it is incorrect. When you return 0 or 1 from svmclassify, you maybe have several zeros or ones with different classes!!!

I think you need to compute the real values(before execute sign function) and assign the test data to the class which has the biggest positive ouput.

LulinThis code can be improved if it can determine a class membership when a test observation belongs to multiple classes.

rajithanks for the updated contents!!! very much helpful...

joy barbosathank you so much for sharing your code sir Cody. It saved my day!!! 2thumbs up!

joy barbosaMatthew KlingensmithB mohanI am doing project on facial expression recognition using LBP(Local Binary pattern). i.e finding expressions whetherit is fear, anger, surprise, disguist, happy, joy. First I loaded jaffee database. Then applied preprocessing on it. And the features are extracted using LBP(Local Binary pattern). I don't know how to use multiclass SVM(Support vector Machine). Can this code be useful to my project. What shoud I take input TrainingSet ,GroupTrain,TestSet. Please help me.

B mohanI am doing project on facial expression recognition using LBP(Local Binary pattern). i.e finding expressions whetherit is fear, anger, surprise, disguist, happy, joy. First I loaded jaffee database. Then applied preprocessing on it. And the features are extracted using LBP(Local Binary pattern). I don't know how to use multiclass SVM(Support vector Machine). Can this code be useful to my project. What shoud I take input TrainingSet ,GroupTrain,TestSet. Please help me.

NimmiCan this be used in model predictive control algorithm.

SubhaSir is it possible for us to add weights for the feature with this code??? if so how to proceed with it..?

SubhaThank you sir.. now trying out with best possible combination...

CodyIf you are getting all 1s, you should try using different options for the svmtrain function. for example:

options = statset('MaxIter',2000000);

models(k) = svmtrain(X_Train,G1vAll,'options',options,...

'kernel_function','rbf','rbf_sigma',16);

Subhasir,

is it possible to plot confusion matrix for this program... if so how to do tat.. we have tried many combinations but does not working...

....

else any method to calculate the performance of the overall system from test data's..

Subhaoh.. ok sir.. now its working.. thank a lot...

CodyOk, you are not using the function correctly.

load your .mat file, then pass your new variables as parameters to the function.

SubhaSir,

I've loaded data and worked in 2 class problem. its works well...

this is my code.. i've just loaded the data and didnt change anything more.

....

function [result] = multisvm(TrainingSet,GroupTrain,TestSet)

load TrainingSet

load GroupTrain

load TestSet

u=unique(GroupTrain);

numClasses=length(u);

result = zeros(length(TestSet(:,1)),1);

%build models

for k=1:numClasses

%Vectorized statement that binarizes Group

%where 1 is the current class and 0 is all other classes

G1vAll=(GroupTrain==u(k))

models(k) = svmtrain(TrainingSet,G1vAll,'kernel_function','rbf');

end

%classify test cases

for j=1:size(TestSet,1)

for k=1:numClasses

if(svmclassify(models(k),TestSet(j,:)))

break;

end

end

result(j) = k;

end

where,

training set is 16*4

test set is 4*4

grouptrain is 16*1

I'm using Matlab (R2010a)

CodyIt sounds like you need to learn how to load .mat files. Simply type 'load xxx.mat' where xxx is the file name. You would then need to sample your data into training, test, and label sets for use with this function.

Can you show the code you are using?

SubhaHello Sir,

Thanks for the code.. it works well when we give data... but i'm working with large set of data, where i need to load it(.mat)..

I cant load data, if i do so then it showing error message

??? Undefined function or variable 'GroupTrain'.

Error in ==> neuro_multisuport at 40

u=unique(GroupTrain);

where grouptrain is a column vector(16*1) in .mat format

.. can u please suggest me how to modify the code when i load data...

NiyalatulHello,

Thanks for your program. I am still new to learn this topic. This is very helpful

NiyalatulNiyalatul