Does the type of classifier make that much difference in feature selection in sequentialfs function?

3 views (last 30 days)
Hi,
I am wondering if does it make significant difference in feature selcetion if we are using difference classifiers during feature selection?
For example, I am using the following naive bayes classifier:
fun = @(Xtrain,Ytrain,Xtest,Ytest)...
sum(Ytest~=predict(fitcnb(Xtrain,Ytrain,'Distribution','kernel'),Xtest));
[inmodel,history] = sequentialfs(fun,myData,classes,'cv',c2,'options',opts,'nfeatures',60);
or if I use fitsvm, instead of fitcnb, will make that much difference on the selceted features?
Thanks

Answers (1)

Don Mathis
Don Mathis on 4 Jan 2019
I think in general it will make a difference which classifier you use, because different classifiers deterct different kinds of patterns. An SVM could discover that a pair of uncorrelated predictors are both relevant because it can learn nonlinear I/O relationships, while a linear classifier might not be able to detect that nonlinear structure, and then "decide" that both predictors are irrelevant.
  2 Comments
Sara Salimi
Sara Salimi on 10 Jan 2019
Thank you very much for your help,
So, can I come to this conclusion that for finding the relevant features, it would be better to use SVM?
Don Mathis
Don Mathis on 10 Jan 2019
I don't think there's any right answer here. You could even try both and take the union of the features. The features you get will also depend on the hyperparameters you choose for your SVM or NB models.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!