4.48649

4.5 | 38 ratings Rate this file 219 Downloads (last 30 days) File Size: 297 KB File ID: #14888

Mutual Information computation

by

 

06 May 2007 (Updated )

A self-contained package for computing mutual information, joint/conditional probability, entropy

| Watch this File

File Information
Description

A self-contained, cross-platform, package for computing mutual information, joint/conditional probability, entropy, and more. This package has also been used for general machine learning and data mining purposes such as feature selection, Bayesian network construction, signal processing, etc.

Another related package for minimal redundancy feature selection is also available at the Matlab Central exchange site, under the category of "Biotech and Pharmaceutical".

A simple demo is called demo_mi.m.

*** Please note that downloading or use of this package means acceptance of the license of this package. In short, this package is free to non-profit use but cannot be re-distributed in any form, including revised forms, without the explicit permission for the author, Hanchuan Peng. See readme file for further information. ***

MATLAB release MATLAB 7 (R14)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (70)
01 Sep 2014 Adam  
12 Aug 2014 alireza

this code not run at all and nobody answer this problem!!!!thanks really

05 Jul 2014 Mohammad Haghighat  
23 Jun 2014 saisakul C

Hi All,

For those who can't run the demo file because of 'undefined function' error. You need to:

1) Run this command: list = dir('*.cpp');
to get the list of files.

2) For all the files in list, change log(2) to log(2.0). You can get all the files' names by:

for i=1:length(list)
list(i).name
end

2) Run 'makeosmex.m'

Now this should works like a dream.

06 Jun 2014 Nawaz Pasha

I got following error, on running demo_mi.m

Undefined function 'estpab' for input arguments of type 'double'.

Error in mutualinfo (line 21)
[p12, p1, p2] = estpab(vec1,vec2);

Error in demo_mi (line 25)
mutualinfo(a,b)

What's wrong ?

02 Jun 2014 Hang Zeng

I also get the problem "Undefined function 'estpa' for input arguments of type 'double'". I find the estpa is cpp file which is a C++ file. I use Win8 62 bit application and Matlab 2011b. What's wrong with it?

28 May 2014 Wadim

when the range of the features vectors is large, let's say 1-10000 estpab() calculates probibilty density with the same length (10000 in this case). Now if both feature vectors have a large range, lets say the other one is 1-10000 too, then the joint pdf computed by estpab is a 10000 x 10000 matrix and the computation if MI takes very long time. Is there a way to modify the algorithm in order to avoid this problem?

25 Apr 2014 Affan

I am getting following error message when I running mi software package:
even I am running this mi package on Intel core i5 with 8 GB memory.

----------------
??? Error using ==> estpab
Out of memory. Type HELP MEMORY for your options.

Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> simple_main_MISS_AR at 174
mi_1(i)= mutualinfo(feat_2,e_1(i,:));
--------------

feat_2 and e_1(i,:) are 1x600 vectors.

please help regarding this.

Thanks

Affan

17 Feb 2014 Douglas Jones

I also got the ESTPAB problem. When I tried running makeosmex to recompile mex files I got the error below.

>>
xcodebuild: error: SDK "macosx10.7" cannot be located.
xcrun: error: unable to find utility "clang++", not a developer tool or in PATH

mex: compile of ' "estcondentropy.cpp"' failed.

Unable to complete successfully.

Error in makeosmex (line 25)
mex(list(i).name);
<<

None of the solutions on this board worked for me (e.g. changing log(2) to log(2.0) or to log(double(2)), etc.

The problem turned out to be that I have Xcode 5.0 and Mac OS10.8, which apparently does not work out-of-the-box. The solution suggested by the Mathworks support team worked for me:
http://www.mathworks.com/matlabcentral/answers/103904

After that, makeosmex compiled for me and I was able to run demo_mi.

Other than that this package looks good.

24 Jan 2014 Anamika

I am getting a problem "Undefined function 'estpa' for input arguments of type 'double'". I use Win7 32 bit application. Actualy what is the problem?

02 Dec 2013 robert marquez  
30 Nov 2013 Yongning

Is there any assumption made when calculating mutualinfo(x, y) if one of vector x,y is binary?
In my data, y is binary and I tried several different x. Surprisingly, several different x give the same mutualinfo to the last digit of double type.
I think some assumption must be made on estimating kernel density. Could author give some insight?

10 Jun 2013 Lei Yang

good

03 May 2013 Pedro Silva

btw, I'm running Matlab 64-bit with linux, which process limit is 8 TB

05 Mar 2013 Davis

Maybe I'm missing something, but I think this code produces weird behavior with vectors of low values, e.g. returning entropy of 0 for [1:10] * .0001.

Run this code and see for yourself:
a = [];
for i = 1:20e4
a(i) = entropy([1:10] .* i/10e4);
end
plot(a)
set(gca,'xticklabel',[0:.2:2])
ylabel('Est. Entropy')
xlabel('Scaling factor')

08 Jan 2013 Jing Wang

Thanks you all. For win 7 64-bit version, the contents of all .cpp files should be changed from log(2) to log(2.0) if log(2) is used.

03 Jan 2013 Gordon

ESTPAB problem.

The mex dlls compiled I believe are 32bit. If you have a 64bit MATLAB application you will not be able to run this software.

To solve this

1) Run makeosmex.m found in the source directory "mi".

2) This failed for me as the log functions in the c++ files needed to be adjusted.

3) For each failure, find the line in the .cpp file where the mex compiler falls over (this will be output to the matlab command window) and replaced log(2) with log(2.0)

Best of luck.

13 Dec 2012 Tripti Malviya

what is condvec in the condmutualinfo.m file???Can anybody please help me?

10 Dec 2012 yawai tint

I have some problem for using this code.When I run the program, I have the error "Invalid mex-file: the ..........estentropy.dll file isn't a valid win64 application".
Could somebody explain to me what is the problem?
I am using the matlab version (R2010) on a windows machine.
Thank you very much!

16 Jul 2012 khalil ben  
20 Jun 2012 May

Thank you

05 Jun 2012 Stefan

function 'mutualinfo' can not find function 'estpab'

where is the problem?

06 May 2012 Onur Ugurlu

I wanted to try say,
how can I change the time delay of mutual function ??

THanks...

03 May 2012 Onur Ugurlu

Hi everyone,
How can ı add decay function or if it already in the program, how can ı can chanhe its valuse.?
if someone can help me, he will save my life :)
Thanks..

15 Apr 2012 Mohammed Ambu

Thank you

23 Mar 2012 Ryan

Does anyone have a recommendation on how to compute entropy conditioned on multiple variables e.g. H(X|Y,Z,W)?

Thanks!

20 Mar 2012 Wei Xiong

I want to say thank you so much. You saved my day.

01 Feb 2012 Vincent

For everyone with error C2668 on Windows:

This can be fixed with a simple typecast of the kind:

log(2) -> log(double(2))

Just replace your source code accordingly.

13 Jan 2012 Zeinab

I'm working in Matlab. One file, estpab.m, is missing. Anybody knows what to do?
Thanks

01 Dec 2011 BUCT

thank u

04 Nov 2011 Waljinder Singh

Hi FEUP, As written above, follow the instructions. What OS do u use? Windows, Linux..etc.? You need to install the compatible compiler (if u don't have one)to make OS compatible mex files
-> Also the Rows in the Data [MXN] and the Rows in the class vector [MX1] should be same.

19 Oct 2011 FEUP

Hi! Thanks for this work.
The program works well, but I can't find m-file "estpab" that is use in the m-file "mutualinfo". Do you help me?
(Sorry for my english. :))

11 Oct 2011 Waljinder Singh

Ok it works now for me. I will tell how it worked. Firstly download and install the compatible compilers (1. Microsoft visual 2.Microsoft Windows SDK - for windows 64 bit) according to your Matlab version. Once it is installed on your machine. Type mex -setup in Matlab. Choose the installed version and confirm by typing y. Now make sure you have all cpp extension and header files.
Next work is to edit the cpp extension files. Open the files one by one and where you find log() type double like this: log(double()) and save. Now go in the directory where you have the m file makeosmex and run it. After this add all the files and folders to the current working directory. Run the mRMR MID/MIQ/BASE files and et voila its done !
If any body has more questions you are free to ask

04 Sep 2011 Anatoly

Sorry,

I believe entropy(rand(10000,1)) should be high, cause here we have much uncertainty of values of variable.

entropy(sin(rand(10000,1))) should definitely equal entropy(cos(rand)), and, I suppose, be small, cause we have less information than with pure random signal (albeit also random)?

Can anyone clarify, please?

04 Sep 2011 Anatoly

Good day! thanks for interesting program. Can anyone please explain me why, having entropy(rand(10000,1))=0 (it's ok),
it gives entropy(sin(rand(10000,1)))=0.9978, BUT entropy(cos(rand(10000,1)))=0
???
:-)
I believe entropy(sin(rand(10000,1))) should also equal 0... or not? with cos(rand) or sin(rand) signals we definitely have less information than with pure rand signal?

13 Jun 2011 Sonali

I am getting following errors. I tried to recompile it but still getting these errors.

??? Error using ==> estpab
The two vectors/images should have the same length.

Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> mrmr at 50
t(i) = mutualinfo(d(:,i), f);

12 Jun 2011 Brahim HAMADICHAREF

Although this is a fast routine, the Mutual Information value for small length vector does not seem to be very reliable when using the same vector as X and Y.
Trying the following small test code ...

clc
clear all
close all

nN = 5000;

fMIacc = zeros(1, nN);

figure
hold on
for i = 1:nN
fVec = rand(1,i);
fMI = mutualinfo(fVec,fVec);
fMIacc(1, i) = fMI;
end
plot(fMIacc)
axis([-100 nN+1 0 1.5])

02 May 2011 HONGJING

I solved my problem by recompile everything.

02 May 2011 HONGJING

Actually I have exactly the same problem as Jochen Kumm cause I'm using Mac. I don't know how to fix it.

01 Mar 2011 Jochen Kumm

I am running matlab on mac osx 10.4 (matlab 7.7.0 R2008b) and also on the unix server (matlab 7.11.0 R2010b), would you please tell me how I can compile the files on both of these machines.

I could run makeosmex.m on the unix machine without any error but when I tried to run demo_mi.m, it gave me the following error (I actually replaced log(2) with log(2.0) in estmutualinfo.cpp as well):

??? Undefined function or method 'estpab' for input arguments of
type 'double'.

Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> demo_mi at 25
mutualinfo(a,b)

On mac osx I even could not run makeosmex.m, it gave me the following error:

building mex(dll) of estcondentropy.cpp
mex(list(i).name);
/Applications/MATLAB_R2008b.app/bin/mex: line 1026: gcc-4.0: command not found
/Applications/MATLAB_R2008b.app/bin/mex: line 1005: gcc-4.0: command not found
/Applications/MATLAB_R2008b.app/bin/mex: line 1: g++-4.0: command not found

mex: compile of ' "estcondentropy.cpp"' failed.

??? Error using ==> mex at 213
Unable to complete successfully.

Error in ==> makeosmex at 25
mex(list(i).name);

I would really appreciate it if you could help me compile the files and execute mutualinfo.m, it is my first time compiling a C source code on matlab and I have no idea how it works. I would be grateful if you could guide me step by step. Thank you for your time.

please send your answer as a comment on this page.

14 Jan 2011 Md. Ali Hossain

Hello, I do not understand the parameter d and f clearly. What will be the value of d? is it individual band for a hyperspectral image, and what will f then? Please inform me.Its better if one can send me with some example data set.

Thanks
Ali

14 Jan 2011 Md. Ali Hossain

Hello, I o not understand the parameter d and f clearly. What will be the value of d? is it individual band for a hyperspectral image, and what will f then? Please inform me.

Thanks
Ali

19 Dec 2010 Ali Farzan

I have the same problem as Benjamin Mack.
How can I fix it?

17 Dec 2010 Vishvas Vasuki

Tried mutual information computation code. Works well!

28 Oct 2010 Jong Rah  
24 Sep 2010 Nanye

I was using mutualinfo() and found the following errors

pure virtual method called
terminate called without an active exception
mutualinfo:778(<D,M> <D,C>):(46912990672832, 2433) -> (<T,T>)

Any hint? Thanks!

17 Aug 2010 Mihály

Hi,
I re-compiled the cpp files with mex to match te current lib versions, but when I compute a mutual information, I always get 0. Did anybody encounter this problem? (I use Linux)

Thanks.

31 Mar 2010 wang

Hi, Marie, have you compiled the estentropy.cpp on your computer?

29 Mar 2010 Marie

When I run the program, I have the error "Invalid mex-file: the ..........estentropy.dll file isn't a valid win32 application".
Could somebody explain to me what is the problem?
I am using the matlab version (R2009) on a windows machine.
Thank you very much!

18 Mar 2010 wang

I find the problem.
define "pabhei" "pabwid" with "long" is ok.
about line 32-34 in "estcondentropy.cpp" .

Is it right?

18 Mar 2010 wang

Hi, Prof Peng,

I still met one problem when I compile "estcondentropy.cpp" by run "mex estcondentropy.cpp" after I changed log(2) to log(2.0).

The other files can be compiled successfully. Can you help me for compiling "estcondentropy.cpp"?

Thank you!
Simbest Wang

16 Mar 2010 chetana nayaka

thank u for the toolbox...

28 Feb 2010 Omid Aghazadeh

I'm sure the log errors are solved by now. It is obvious that you need to add a .0 to the log(2) to solve the problem if you get that error. (change log(2) to log(2.0)).

25 Feb 2010 Benjamin Mack

Hello,
I try to use this programm but I am totally new to matlab (but have experience in R). When I call the function
mrmr_mid_d (variables, response, 25)
the following error occurs:
??? Undefined function or method 'mrmr_mid_d' for input arguments of type 'double'.
The same occures when I convert my data in int8 or single.

something similar happens when I try to use the demo_mi.m :

mutualinfo(a,b)
??? Undefined function or method 'estpab' for input arguments of type 'double'.
Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

I think my problem is very a very basic one because I do not know anything about Matlab.
Could anybody please help me. It is very important for me to use this programm and the online version does not work with my data set because it is too large.

Thank you very much in advance,
Ben

02 Feb 2010 Raphael

I am trying to compile mex files on Unix with Matlab 7.7.0(2008b) using makeosmex.m. I get the following error messages:
building mex(dll) of estcondentropy.cpp
estcondentropy.cpp:19:23: error: miinclude.h: No such file or directory
estcondentropy.cpp:21: error: ?mxArray? has not been declared
estcondentropy.cpp:21: error: expected ?,? or ?...? before ?*? token
estcondentropy.cpp:21: error: ISO C++ forbids declaration of ?mxArray? with no type
estcondentropy.cpp: In function ?void mexFunction(int, int**, int, int)?:
estcondentropy.cpp:24: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:26: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:32: error: ?prhs? was not declared in this scope
estcondentropy.cpp:32: error: ?mxGetPr? was not declared in this scope
estcondentropy.cpp:33: error: ?mxGetM? was not declared in this scope
estcondentropy.cpp:34: error: ?mxGetN? was not declared in this scope
estcondentropy.cpp:41: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:59: error: ?log? was not declared in this scope
estcondentropy.cpp:65: error: ?log? was not declared in this scope
estcondentropy.cpp:67: error: ?mxREAL? was not declared in this scope
estcondentropy.cpp:67: error: ?mxCreateDoubleMatrix? was not declared in this scope

Suggestions?!

12 Nov 2009 Carlos G-M

I can't compiler the file estmutualinfo.cpp.
estmutualinfo.cpp
estmutualinfo.cpp(42) : warning C4267: 'initializing' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(43) : warning C4267: 'initializing' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(55) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(58) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(65) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(67) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(120) : error C2668: 'log' : ambiguous call to overloaded function
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(567): could be 'long double log(long double)'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(519): or 'float log(float)'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(121): or 'double log(double)'
while trying to match the argument list '(int)'

30 Sep 2009 Hanchuan

Quite some people ask for mex files for different operating system versions, such as the latest Windows, or Mac OS X 10.6 Snow Leopard, etc. Please note that the mex functions can simply be re-generated by running the "makeosmex" command when you enter the folder of this toolbox (I assume you know how to set up mex compiling environment). For Windows, I use mingw to compile.

27 Dec 2008 Bahaa Khalil

Many thanks for your contribution, is it possible to adjust estpob file to be able to calculate mutual information for continuous data

26 Sep 2008 Yan Jin

Thank you for your source code! It's very good^^

25 Jul 2008 ming ma

very good

28 Apr 2008 Guenther Eibl

I just needed the normalized MI which was easy to find and use

16 Feb 2008 Usha Nair

i want to normalise the mutualinfo to the range (0-1) for comparison of different time series. Could you please help me by proper suggestion for this. Is it possible to change the number of bins used for the histogram so that all the data sets can be analyzed using the same scale.

03 Feb 2008 Dedy H

Can you help me to find joint probability for multivariate data....???how can I contact with you???

23 Jan 2008 sumit maheshwari

i want to write joint probabilty as a function or want to express as an equation....
please help me.......

11 Jan 2008 Fernando Diaz

I need un function that is needed in mutualinfo (estpab)

18 Dec 2007 Mike B  
29 May 2007 Rui Liu

very good!!

22 May 2007 Peter Chow

a nice piece of work, make my life much easier!

07 May 2007 ali shrof  
07 May 2007 Xiang Y

great! easy to use.

Contact us