Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
How can I train SVM in Matlab, with svmtrain command, but for more

Subject: How can I train SVM in Matlab, with svmtrain command, but for more

From: Eizo

Date: 17 Dec, 2009 17:02:31

Message: 1 of 10

Hi
I was playing with the option to implement SVM in my OCR algorithm,
but when checking the SVM that Matlab has to offer I saw that it can
support only 2 groups training and classification ?
Am I am missing something ?

I was using the svmtrain command, and tried to train it for 3 groups
(e.g. three characters.) and I got the error saying that svm train
support only two groups...

Am I am missing something in the SVM implementation ? or it is really
that Matlab support only two groups, and when you want to implement
something bigger then that, like ocr for 10 characters (e.g. 10
groups) , matlab doesn't support that (at least in it's ready made
tool box).

Thanks
O

Subject: How can I train SVM in Matlab, with svmtrain command, but for more

From: Bruno Luong

Date: 17 Dec, 2009 19:03:08

Message: 2 of 10

Eizo <odperry@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f25a1@d10g2000yqh.googlegroups.com>...

> Am I am missing something ?

SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).

There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.

Bruno

Subject: How can I train SVM in Matlab, with svmtrain command, but for more

From: Pekka Kumpulainen

Date: 17 Dec, 2009 19:20:08

Message: 3 of 10

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hgdv9c$o9q$1@fred.mathworks.com>...
> Eizo <odperry@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f25a1@d10g2000yqh.googlegroups.com>...
>
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
>
> Bruno

At least the SVM in the Bioinformatics toolbox does only two-class classification.
There are other tools available. LIBSVM for example supports multi-class and has MATLAB interface
http://www.csie.ntu.edu.tw/~cjlin/libsvm/

Subject: How can I train SVM in Matlab, with svmtrain command, but for

From: Eizo

Date: 18 Dec, 2009 09:13:54

Message: 4 of 10

On Dec 17, 9:03 pm, "Bruno Luong" <b.lu...@fogale.findmycountry>
wrote:
> Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>...
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
>
> Bruno

Then how people use SVM for face recognition or OCR, if it is just 2
class learning technique?
(They use the multi class trick ? )

op

Subject: How can I train SVM in Matlab, with svmtrain command, but for

From: Eizo

Date: 18 Dec, 2009 17:56:08

Message: 5 of 10

On Dec 17, 9:03 pm, "Bruno Luong" <b.lu...@fogale.findmycountry>
wrote:
> Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>...
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
>
> Bruno

Where can I read on the Multi-class methods? .. maybe you know on
some links to some tutorials on that matter ?
All the papers and presentation I read deal only with 2 classes..

Subject: How can I train SVM in Matlab, with svmtrain command, but for more

From: Ilya Narsky

Date: 18 Dec, 2009 18:35:16

Message: 6 of 10


"Eizo" <odperry@gmail.com> wrote in message
news:dd4af2f0-f645-4576-8713-d3fe746d613b@l13g2000yqb.googlegroups.com...
On Dec 17, 9:03 pm, "Bruno Luong" <b.lu...@fogale.findmycountry>
wrote:
> Eizo <odpe...@gmail.com> wrote in message
> <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>...
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just
> that. For multi-classes you might want to apply successively 2-classes
> learning processes. See relevant papers for how to use correct strategy
> ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse
> classification, but I don't think Matlab has any of those technique.
>
> Bruno

Where can I read on the Multi-class methods? .. maybe you know on
some links to some tutorials on that matter ?
All the papers and presentation I read deal only with 2 classes..

Allwein, Schapire and Singer "Reducing multiclass to binary: a unifying
approach for margin classifiers", J. of Machine Learning Research 1 (2000)
113-141

Kuncheva "Using diversity measures for generating error-correcting output
codes in classifier ensembles", Pattern Recognition Letters 26 (2005) 83-90;
also references therein

Tibshirani and Hastie "Margin trees for high-dimensional classification",
available from http://www-stat.stanford.edu/~hastie/

For SVM:

Lee, Lin and Wahba "Multicategory support vector machines: theory and
application to the classification of microarray data and satellite radiance
data", J. of ASA 99 (2004)

Subject: How can I train SVM in Matlab, with svmtrain command, but for more

From: Bruno Luong

Date: 18 Dec, 2009 19:01:24

Message: 7 of 10

- Analysis of Multiclass Support Vector Machines, Shigeo Abe

- Winner takes all -> B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In KDD'95, 1995.

- One againsts one -> S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited : A stepwise procedure for building and training a neural network. In F. Fogelman-Soulié and J. Hérault, editors, Neurocomputing: Algorithms, Architectures and Applications, volume F68 of NATO ASI Series, pages 41-50. Springer-Verlag, 1990.

- Decision graph: J.C. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGs for multiclass classication. In NIPS 12, pages 547-553, 2000.

Bruno

Subject: How can I train SVM in Matlab, with svmtrain command, but for

From: Eizo

Date: 26 Dec, 2009 16:27:47

Message: 8 of 10

On Dec 18, 9:01 pm, "Bruno Luong" <b.lu...@fogale.findmycountry>
wrote:
> - Analysis of Multiclass Support Vector Machines, Shigeo Abe
>
> - Winner takes all -> B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In KDD'95, 1995.  
>
> - One againsts one -> S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited : A stepwise procedure for building and training a neural network. In F. Fogelman-Soulié and J. Hérault, editors, Neurocomputing: Algorithms, Architectures and Applications, volume F68 of NATO ASI Series, pages 41-50. Springer-Verlag, 1990.
>
> - Decision graph: J.C. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGs for multiclass classi cation. In NIPS 12, pages 547-553, 2000.
>
> Bruno

While exploring the net for some Matlab code that can do multiclass
SVM. I was wondering, is it possible to train the SVM with a character
(e.g one vector that contain binary letter information, with 1's and
0's) like in the neural net training ?

Because all the demos, and all the articles I read deal with 2 dim
info.
While it is possible to find some features that describe letters in
the 2 dim space, I was looking to speed things up, by entering the
letter as is in its raw binary info (like neural net).

Thanks.

Subject: How can I train SVM in Matlab, with svmtrain command, but for

From: roshani.ade@raisoni.net

Date: 15 Jan, 2013 07:22:24

Message: 9 of 10

On Thursday, December 17, 2009 10:32:31 PM UTC+5:30, Eizo wrote:
> Hi
> I was playing with the option to implement SVM in my OCR algorithm,
> but when checking the SVM that Matlab has to offer I saw that it can
> support only 2 groups training and classification ?
> Am I am missing something ?
>
> I was using the svmtrain command, and tried to train it for 3 groups
> (e.g. three characters.) and I got the error saying that svm train
> support only two groups...
>
> Am I am missing something in the SVM implementation ? or it is really
> that Matlab support only two groups, and when you want to implement
> something bigger then that, like ocr for 10 characters (e.g. 10
> groups) , matlab doesn't support that (at least in it's ready made
> tool box).
>
> Thanks
> O

Subject: How can I train SVM in Matlab, with svmtrain command, but for

From: Alan_Weiss

Date: 15 Jan, 2013 12:48:48

Message: 10 of 10

On 1/15/2013 2:22 AM, roshani.ade@raisoni.net wrote:
> On Thursday, December 17, 2009 10:32:31 PM UTC+5:30, Eizo wrote:
>> Hi
>> I was playing with the option to implement SVM in my OCR algorithm,
>> but when checking the SVM that Matlab has to offer I saw that it can
>> support only 2 groups training and classification ?
>> Am I am missing something ?
>>
>> I was using the svmtrain command, and tried to train it for 3 groups
>> (e.g. three characters.) and I got the error saying that svm train
>> support only two groups...
>>
>> Am I am missing something in the SVM implementation ? or it is really
>> that Matlab support only two groups, and when you want to implement
>> something bigger then that, like ocr for 10 characters (e.g. 10
>> groups) , matlab doesn't support that (at least in it's ready made
>> tool box).
>>
>> Thanks
>> O

Currently, SVM supports only binary (two-group) classification. However,
there are other Statistics Toolbox functions that support that support
multiple classes, such as discriminant analysis
http://www.mathworks.com/help/stats/discriminant-analysis.html
classification trees
http://www.mathworks.com/help/stats/classification-trees-and-regression-trees-1.html
nearest neighbor
http://www.mathworks.com/help/stats/classification-using-nearest-neighbors-1.html
and ensemble methods
http://www.mathworks.com/help/stats/ensemble-methods.html#btgalua

Alan Weiss
MATLAB mathematical toolbox documentation

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us