version 1.0.0.0 (523 KB) by
Hanchuan Peng

This is a cross-platform version of mimimum-redundancy maximum-relevancy feature selection

66 Downloads

Updated 19 Apr 2007

No License

This package is the mRMR (minimum-redundancy maximum-relevancy) feature selection method in (Peng et al, 2005 and Ding & Peng, 2005, 2003), whose better performance over the conventional top-ranking method has been demonstrated on a number of data sets in recent publications. This version uses mutual information as a proxy for computing relevance and redundancy among variables (features). Other variations such as using correlation or F-test or distances can be easily implemented within this framework, too.

Hanchuan Peng, Fuhui Long, and Chris Ding, "Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy,"

IEEE Transactions on Pattern Analysis and Machine Intelligence,

Vol. 27, No. 8, pp.1226-1238, 2005. [PDF]

Ding C., and Peng HC, "Minimum redundancy feature selection from microarray gene expression data," Journal of Bioinformatics and Computational Biology,

Vol. 3, No. 2, pp.185-205, 2005. [PDF]

Ding, C and Peng HC, Proc. 2nd IEEE Computational Systems Bioinformatics Conference (CSB 2003),

pp.523-528, Stanford, CA, Aug, 2003.

Hanchuan Peng (2020). mRMR Feature Selection (using mutual information computation) (https://www.mathworks.com/matlabcentral/fileexchange/14608-mrmr-feature-selection-using-mutual-information-computation), MATLAB Central File Exchange. Retrieved .

1.0.0.0 | correct some typos |

Created with
R14SP3

Compatible with any release

**Inspired:**
Backpropagation-based Multi Layer Perceptron Neural Networks

Create scripts with code, output, and formatted text in a single executable document.

Xiao WuSreevalsan Menon@Lorenz Ott https://www.mathworks.com/help/matlab/ref/mex.html

Lorenz Ott@Mubashir Thank you you very much for your explanation!

I don't get your points 3 and 4 ("3. mex -setup, 4. makeosmex").

What do you mean by "mex"?

Thanks in advance!

Indy MagnusDZARITH AMANDA BINTI ABDUL RAHMANwheres the code btw? how can i download?

MariannaIIs there a limit for the input arrays? Because every time I try to run the code, Matlab simply crushes and I have to close it.

I run:

[fea] = mrmr_mid_d(FEAT, score, 30)

where FEAT is a 30125x656 array and score is a 30125x1 array

When I try on a super small sample, it works perfectly.

I might have an other question: have you anywhere specified what exactly fea is? Is it an array with the indexes of the extracted features?

Thank you very much for the help.

Winston Zhang@Karakule I also got this error, you can fix it by quantizing your data before inputting into mRMR function, since the mRMR algorithm requires discrete bins. You can use the discretize MATLAB function on every feature column in your data, the number of bins used is a hyperparameter that usually is set to between 3 and 10.

Chris LAUmi folder under toolbox should be selected as Current Folder in Matlab.

Chris LAUI have copied the file estpab.cpp from "https://in.mathworks.com/matlabcentral/fileexchange/56723-yan-prtools" to the toolbox folder in my Matlab. And then executive "makeosmex" again. Wow, it does work.

Thank you all for your help.

KarakuleI did eveything in comments. But I am also getting below error again. My code is given below;

d=data(1:2100,1:29);

d = double(d);

f=data(1:2100,30:30);

f = double(f);

mrmr_mid_d(d,f,5) %for selecting 5 features.

Could you help me please?

Error using estpab

Requested 2145698427x21 (335.7GB) array exceeds maximum array size preference. Creation of arrays

greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size

limit or preference panel for more information.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_mid_d (line 14)

t(i) = mutualinfo(d(:,i), f);

MubashirIt took a day to find a solution from given comments.

1) Undefined function or variable 'estpab':

Solution:

1. Open "miinclude.h" and change:

#include <math.h> to #include <cmath>

2. Open "estjointentropy.cpp" "estentropy.cpp" "estmutualinfo.cpp" "estcondentropy.cpp" and change log(2) to log(2.0)

3. mex -setup

4. makeosmex

And it works!

Note: if you can't find .cpp files download 'Mutual Information computation' from here https://www.mathworks.com/matlabcentral/fileexchange/14888-mutual-information-computation

Replace mi folder with a downloaded toolbox.

If you find error in mex compiling

1) I tried different compiler ‘tdm-64’ using https://sourceforge.net/projects/tdm-gcc/files/latest/download?source=typ_redirect link.

2) After installing set variable ‘setenv('MW_MINGW64_LOC','C:\TDM-GCC-64')’

3) mex -setup

4) makeosmex

Last command syntax should be:

mrmr_mid_d(all features,labels,important features)

e.g. mrmr_mid_d(findata(:,1:60),findata(:,60),15)

shaheda banohttps://in.mathworks.com/matlabcentral/fileexchange/56723-yan-prtools

This link contain all the necessary files related to this topic.It also contain 'estpab'.

Have a look!

Hosein BabajanianJavaid IqbalI'm facing still this problem, I have modified as mentioned in comments please help it will be very appreciative.

Undefined function or variable 'estpab'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 16)

t(i) = mutualinfo(d(:,i), f);

monika roopakI am having following error, could anyone help with this

Error in entropy (line 35)

[p1] = estpa(vec1);

puppythe result of the demo：

mutualinfo(a,b)

ans =

0.4200

entropy(a)+condentropy(b)-jointentropy(a,b)

ans =

0.4200

condentropy(a,b)

ans =

0.5510

condentropy(a,c)

ans =

0.9510

jointentropy(a,c)

ans =

1.9219

mutualinfo(a,c)

ans =

0.0200

condmutualinfo(a,c)

ans =

0.0200

condmutualinfo(a,c,b)

ans =

0.1510

condmutualinfo(a,c,[b c])

ans =

0

echo off;

SAMSON PAccepted Answer

How to fix "estpab" issue:

1. Open "miinclude.h" and change:

#include <math.h> to #include <cmath>

2. Open "estjointentropy.cpp" and "estcondentropy.cpp" and change (row 56 | row 65): "muInf /= log(2);" to "muInf /= log(2.0);"

3. Recompile

changchang有人能分享一下demo的计算结果吗？

changchangAli Khoshrooming jingtide_bamtide_bamI am also facing the same error of "estpab"

how can I slove this question.

这编译好的程序好像不能用啊。= =、怎么解决啊。

yaseen iqbalI am facing the same error of "estpab"

Kindly solve this problem or recommend any alternative Please.

tide_bamtide_bamarwa aliI run the code and got the following error

Undefined function or variable 'estpab'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_mid_d (line 17)

t(i) = mutualinfo(d(:,i), f);

Error in mrmr (line 11)

[fea] = mrmr_mid_d(Features,classLable,5);

Helix HsunabasmitaGffgjh fggfdhI this library support windows 32 bit. I am getting following error.

mrmr_mid_d(d,f,4); d=35x59 matrix f=35x1 matrix

Invalid MEX-file ../mi_0.9\estpab.dll

../mi_0.9\estpab.dll is not a valid Win32 application.

zaafouri rabaacan someone help me how to run this code?

HMMWhen I run makeosmex.m, there is an error that estcondentropy.cpp(65) : error C2668: "log": call of the overload function is unclear.

Is someone can help me solve this problem? thanks!

Muhammad Aqil Mohammad HanafiWhat kind of input arguments I have to input when running the "mrmr_mid_d(d, f, k)"? Is it some kind of data set I have to download from external source?

BAMUwhen i m running

[fea]=mrmr_mid_d(d,f,k)

i got error

Undefined function 'mutualinfo' for input arguments of type 'double'.

Error in mrmr_mid_d (line 17)

t(i) = mutualinfo(d(:,i), f);

delony Lidelony LiUndefined function or method 'estpab' for input arguments of type 'double'.

Anyone have got errors? How to solve it?

Rodrigo CeballosCan someone that has solved the 'estlab issue' just upload a revised version somewhere after they recompile the error? Zhang Yue and Beytula Redzheb for example?

hanu jwhen i am running

[fea] = mrmr_mid_d('wdbc.xlsx', 'class.xlsx', 10);

i check both ways first row as attribute names and with out attribute names

i got errors are below any one help me how to fix these errors

??? Error using ==> estpab

The first input argument must be types of INT8 or UINT8 or

DOUBLE.

Error in ==> mutualinfo at 21

[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> mrmr_mid_d at 17

t(i) = mutualinfo(d(:,i), f);

thanks in advance

Md . Asaduzzaman Asadwho have problem with

"Undefined function 'estpab' for input arguments of type 'double'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in demo_mi (line 25)

mutualinfo(a,b)"

should install visual studio in case of windows os.

TaraMI'm using mac OS X Yosemite. Similar to many other users I ran into the following error:

"Undefined function 'estpab' for input arguments of type 'double'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in demo_mi (line 25)

mutualinfo(a,b)"

I know many have suggested to change somethings in .cpp files but I only can find .mexmac , .mexglx and .dll files. Would someone please help me to figure out what I should be doing? Thanks!

Md . Asaduzzaman Asadwhen i running mrmr_mid_d with d=62x2000 , f=62x1 and K=25 i got the following error:

----------------------------------------

Error using estpab

Requested 12962x19852 (1.9GB) array exceeds maximum array size preference. Creation of

arrays greater than this limit may take a long time and cause MATLAB to become

unresponsive. See array size limit or preference panel for more information.

-------------------------------------

it works fine having d=62x100 ,f=62x1 and K=10 .how to fix it ? Can anyone help plz?

C S KumarMRMR function header says:

Note: This version only supports discretized data, thus if you have continuous data in "d", you will need to discretize them first.

Does it mean that the feature vector elements should be integers, a strange requirement though.

Zhang YueHere is a method to solve the 'estpab' problem.

Open all .cpp files and change 'log(2)' to 'log(2.0)'. Then run 'makeosmex.m' to recompile.

Xinyao TangCould someone to give a brief instruction that how to use this matlab codes. It seems that the files includes mutual information toolbox, however, the error comes out:

Undefined function 'estpab' for input arguments of type 'nominal'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 50)

t(i) = mutualinfo(d(:,i), f);

Beytula RedzhebHow to fix "estpab" issue:

1. Open "miinclude.h" and change:

#include <math.h> to #include <cmath>

2. Open "estjointentropy.cpp" and "estcondentropy.cpp" and change (row 56 | row 65): "muInf /= log(2);" to "muInf /= log(2.0);"

3. Recompile

Lynn Kerhave problem while using mRMR in MATLAB 2014a.

if I run mRMR then error:

the error:

Undefined function 'estpab' for input arguments of type 'nominal'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 50)

t(i) = mutualinfo(d(:,i), f);

I have windows-7 64 bit system and MATLAB version 2014a. Please help .

DSPI have problem while using mRMR in MATLAB 2012b.

if I run mRMR then error:

the error:

Undefined function 'estpab' for input arguments of type 'nominal'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 50)

t(i) = mutualinfo(d(:,i), f);

I have windows-7 64 bit system and MATLAB version 2012b. Please help .

ahmed hamed'm using this algorithm for the first time, but unable to run it.

I'm using matlab 12a

load fisheriris

f = mrmr_mid_d(meas, species, 2)

but i got an error:

undefined function 'mrmr_mid_d' for input arguments of type cell

I also tried another data set where the class is a double matrix and got the same error: undefined function 'mrmr_mid_d' for input arguments of type double

please help.

Emilie MussardThank you for your code.

I followed last comments of sheng (or at least tried to) but I still have a problem with this estpab function, it is notified as "Undefined function 'estpab' for input arguments of type 'int8'"

When it is called by mutualinfo (line 21).

Am I missing something?

shengGreat code.

If you have any problem using this code please check http://penglab.janelia.org/proj/mRMR/FAQ_mrmr.htm#Q1.2

for 'estpab.dll' problem please download this mi version http://www.mathworks.com/matlabcentral/fileexchange/14888-mutual-information-computation

replace it with the folder mi_0.9(Mex compile if needed), then copy the three mrmr files to directory /mi/,and run it with you data（integer） ,label and num of features you want to select.

example: fea = mrmr_mid_d(yourdata,labels,5) %select top 5 features

shengshengGreat code.

If you have any problem using this code please check http://penglab.janelia.org/proj/mRMR/FAQ_mrmr.htm#Q1.2

for 'estpab.dll' please download this mi version http://www.mathworks.com/matlabcentral/fileexchange/14888-mutual-information-computation

replace it with the folder mi_0.9(Mex compile if needed), then copy the three mrmr files run it with you data ,label and num of features you want to select.

zhangwhere to dl？

Mileswhat does that mean？

Warning: Calling MEX-file 'F:\Matlab Process\GEO\mutual

information\mRMR_0.9_compiled\mi_0.9\estpab.dll'.

MEX-files with .dll extensions will not execute in a future version

of MATLAB.

Warning: Calling MEX-file 'F:\Matlab Process\GEO\mutual

information\mRMR_0.9_compiled\mi_0.9\estmutualinfo.dll'.

MEX-files with .dll extensions will not execute in a future version

of MATLAB.

Akram NourI am trying to run mrmr_mid_d.m and mrmr_miq_d.m but an error shows that the file "estpab.m" is missing !!?? you have not answered any of the questions in the posts answering this question :(

Also what is the difference between mrmr_miq and mrmr_mid ?

Please help where can I find this m file

--

the error:

Undefined function 'estpab' for input arguments of type 'nominal'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 50)

t(i) = mutualinfo(d(:,i), f);

--

Akram NourI am trying to run mrmr_mid_d.m and mrmr_miq_d.m but an error shows that the file "estpab.m" is missing !!?? you have not answered any of the questions in the posts answering this question :(

Please help where can I find this m file

--

the error:

Undefined function 'estpab' for input arguments of type 'nominal'.

Error in mutualinfo (line 21)

[p12, p1, p2] = estpab(vec1,vec2);

Error in mrmr_miq_d (line 50)

t(i) = mutualinfo(d(:,i), f);

--

nayeffHi Prof. Peng,

Thank you for the code. I am trying to understand what is the difference between mrmr_miq_d and mrmr_mid_d. Whould you please explain?

Regards,

Yashodhan AthavaleI'm using this algorithm for the first time, and unable to relate the articles published and the code written. As indicated in the function [fea] = mrmr_miq_d(d, f, K):

I did the following:

d= 180 x 48 dataset - indicating 180 samples each having 48 features

f = 180 x 1 - class/category of the n samples

K = 24 - number of features to be selected

I ran the code and got the [fea] as output vector of 1 x 24 dimensions. But what does [fea] mean ? What do the numbers in the fea array signify ?

Jing WangJing Wanghttp://t.cn/zjupBbJ All files and possible problems and answers could be seen in this webpage. Good luck!

LukeI have a program using this mRMR implementation. When I moved my code over from a 32 bit machine to a 64 bit machine and recompiled I receive different feature selected outputs for the same inputs!!!

ZeinabIn order to run mrmr_mid_d.m or mrmr_miq_d.m it needs estpab.m file. Please help where can I find this m file.

Waljinder SinghOk it works now for me. I will tell how it worked. Firstly download and install the compatible compilers (1. Microsoft visual 2.Microsoft Windows SDK - for windows 64 bit) according to your Matlab version. Once it is installed on your machine. Type mex -setup in Matlab. Choose the installed version and confirm by typing y. Now make sure you have all cpp extension and header files. If you don't have them take all from here : http://www.mathworks.com/matlabcentral/fileexchange/14888

Copy all the files which you don't have. Next work is to edit the cpp extension files. Open the files one by one and where you find log() type log(double()) and save. Now go in the directory where you have the m file makeosmex and run it. After this add all the files and folders to the current working directory. Run the mRMR MID/MIQ/BASE files and et voila its done !

If any body has more questions you are free to ask

Chris HallWorks great! Both quick and reliable.

Sangtae AhnWhere is 'estpab.m' file

I cannot find this file to run.

Daniel GoerkeHi, I've got the same memory problem:

I downloaded the mRMR package and furthermore downloaded the recent version of the mutual information toolbox from the autor's website, since the old version was not executable.

I got a data matrix <19900x256 double> and a class_vector matrix <19900x1 double> with values eigther 1, 2, 3 or 4.

I want to compute the "best" 50 features out of the 256.

So what I do now is calling test=mrmr_mid_d(data,class_vector,50);

Now it takes a while (at dual core machine E5300 with 2 GB RAM) and the following error occures:

??? Error using ==> estpab

Out of memory. Type HELP MEMORY for your options.

Error in ==> mutualinfo at 21

[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> mrmr_mid_d at 49

t(i) = mutualinfo(d(:,i), f);

for mrmr_miq_d it's exactly the same.

Any ideas?

Best regards,

Daniel

AirballmanHi, I tried to get the enwest version of the toolbox and recompile all c++ files but it keeps saying me 'Out of memory' just like Maha Sulaeman before.

May someone help me on this?

ZohebThank you Omid. My paths were ok. The problem has been resolved by recompiling the toolbox (this download doesn't contain the source code). It is now working with my feature set.

Omid Aghazadehfor both of the above problems:

you need to have the folder containing estpab in the same folder as mrmr_mi*_d.m . Otherwise, you can use addpath.

The definition of input variables are included in mrmr_mi*_d

ZohebThe inputs are explained on the author's faq here,

http://penglab.janelia.org/proj/mRMR/FAQ_mrmr.htm#s4

"Q4.4 What are the input/parameters of the Matlab version?

A. Three arrays, D, F, and K. D is an m*n array of your candidate features, each row is a sample, and each column is a feature/attribute/variable. F is an m*1 vector, specifying the classification variable (i.e. the class information for each of the m samples). K is the maximal number of features you want to select.

D must be pre-discretized as a categorical array, i.e. containing only integers. F must be categorical, indicating the class information."

Having said that, I'm still struggling myself. The particular error message I'm getting is, "Undefined function or method 'estpab' for input arguments of type 'uint8'."

Could this be due to the data being the wrong type, bad format?

MarieI have the same question as above: what are d (all datas I think), what are f (features? but if you pu features you put all datas...) and K is the number of features we want to select I supposed.

Another question is that I have "Invalid MEX-file... the application .dll is invalid...". Would you know how to solve that.

Thanks a lot for this software anyway!

Omid AghazadehWorked like a charm. Great toolbox for feature selection.

Delinda HeThere is no comment in the code, would you please help me about what is mrmr_mid_d(d, f, K), d? f? K?

Thanks so much.

Maha SulaemanWhenever I use the function mrmr_mid_d, the Matlab keeps displaying 'Out of memory'. my dataset is 202x62 & my RAM space is 2G, does the function need more space? and if so,is there any way I can make it work?

Andre PfeufferI had problem to get this run under R2007a Matlab. When used the libraries from "Mutual Information computation", same author, it get working. Excellent work!

Henry Leecool codes...

harry LeeBob SimpsonSirley KThere is also an online version and another binary executable for different OS at the author's web site: http://research.janelia.org/peng/proj/mRMR/index.htm

ALI EL AKADIX ChenIt may need some efforts to understand what are the d and f, which are the data table and class variable. There is another version at author's web site.

X ChenThis is a quite straightforward implementation and just does what it supposes to do. However it would be nicer if some datasets could be included.