Code covered by the BSD License

### Highlights from Fast mutual information of two images or signals

4.2
4.2 | 10 ratings Rate this file 30 Downloads (last 30 days) File Size: 2.26 KB File ID: #13289 Version: 1.6

# Fast mutual information of two images or signals

### Jose Delpiano (view profile)

07 Dec 2006 (Updated )

Optimized function for mutual information of two images or signals

File Information
Description

Usage: I=mi(A,B), where A and B are equally sized images/signals.
Function hist2 (included) is used to determine the joint histogram of the images/signals.

Assumptions: 1) 0*log(0)=0, 2) mutual information is obtained on the intersection between the supports of partial histograms.

Example (in mi_test.m):

disp('Test: Mutual information between two images')
A=D(:,:,8);
B=D(:,:,9);
mi(A,B)

disp('Test: Mutual information between two signals')
nasdaq = price2ret(NASDAQ);
nyse = price2ret(NYSE);
mi(nasdaq,nyse)

Acknowledgements

This file inspired Image Mi.M.

Required Products Econometrics Toolbox
MATLAB
MATLAB release MATLAB 7.7 (R2008b)
MATLAB Search Path
`/`
28 May 2013 yuting

### yuting (view profile)

Subscripted assignment dimension mismatch.

Error in hist2 (line 32)
n(i+1,:) = histc(B(A==i),x,1);

Error in mi (line 31)
n2 = hist2(A,B,L);

Does anyone have the same problem?

Comment only
10 Mar 2013 German Miroshnichenko

### German Miroshnichenko (view profile)

I think it will be more accurate to use

A=floor((A-ma)*L/(MA-ma+eps));
B=floor((B-mb)*L/(MB-mb+eps));

A=round((A-ma)*(L-1)/(MA-ma+eps));
B=round((B-mb)*(L-1)/(MB-mb+eps));

because round makes the first and the last bins twice less than the others.

Comment only
06 Jan 2013 Bibek

### Bibek (view profile)

on running the code from mi_test.m, I got error message: ??? Undefined function or method 'mi' for input arguments of
type 'uint8'.
I am working in Matlab 7.10.0(R 2010a). I don't know what is the problem. Any help will be highly appreciated.

13 Dec 2012 Heming Liao

### Heming Liao (view profile)

ok. just ignore what i said for the last message. I made some mistake.

Comment only
13 Dec 2012 Heming Liao

### Heming Liao (view profile)

Hi Jose,
Thanks for sharing the code. I think these is still something wrong with the code. when i tried this a = [1,2,3;1,2,3;1,2,3], and calculate mi(a,a), I get 1.0566, so I think there is still something wrong with it. thanks.

17 Jul 2012 Jose Delpiano

### Jose Delpiano (view profile)

Hanbo, many thanks for your comments. I'd like to add this improvement and test it. Would you provide some test data?

Comment only
16 Jul 2012 Hanbo Chen

### Hanbo Chen (view profile)

Thank you for sharing this. I modified based on this to calculate normalized mutual information. However, I found the results could be larger than 1 sometimes (which is wrong). I checked my modified part and everything is correct, which means the mutual information given by this code could be wrong or not accurate. I checked the code and found that it is mainly caused by different definition of bin boundary of hist (matlab defined) and hist2 (user defined). I replaced hist() with following myhist() and the result is now excellent on simulated toy testing data.

function n=myhist(A,L)
ma=min(A(:));
MA=max(A(:));
A=round((A-ma)*(L-1)/(MA-ma+eps));
x=0:L-1;
n=histc(A,x,1);
n=n';
end

04 Nov 2011 scaramanga

### scaramanga (view profile)

Great Job ! However, I'm just wondering why you're using hist in mi function and histc in hist2 function. I would have used histc everywhere. Is there any difference, or it is an arbitrary choice ?

25 Aug 2011 Jose Delpiano

### Jose Delpiano (view profile)

Andrew, many thanks. I'm sorry about my delay. I added the improvements you suggested. The code is much faster now.

Divyesh Varade, mutual information is a measure based on pixel statistics, so it can't be applied pixel by pixel. Applying it to small windows could be a good idea for some applications.

Comment only

Thanks for the code it was very helpful. However, I want to compare two images using MI. If I do it pixel by pixel using the given code, I mostly get a warning: divide by zero and receive a value NaN for MI. Now I can try and work with very small windows also but that would seem to reduce the quality of my results. Is there another way or by enhancing this code somehow to analyse the two images together almost pixelwise.

22 Feb 2010 andrew hill

### andrew hill (view profile)

oh, and somwhere there is a re-scaling bug ; http://www.flickr.com/photos/47796226@N04/4380054294/
which shows the MI between a fixed window in one image, and a sliding window in each of 6 other images.
In some images it works, but in others it doesn't.

Some images have negative intensity; i'll investigate further and post here again if i find the cause. (it happens with both my code and the original, changing the re-binning can slightly change the pattern of the unusual elements.)

Comment only
22 Feb 2010 andrew hill

### andrew hill (view profile)

the current implementation of both MI and hist2 are quite slow;

function n=fast_hist2(A,B,L)
ma=min(A(:));
MA=max(A(:));
mb=min(B(:));
MB=max(B(:));

% For sensorimotor variables, in [-pi,pi]
% ma=-pi;
% MA=pi;
% mb=-pi;
% MB=pi;

% Scale and round to fit in {0,...,L-1}
A=round((A-ma)*(L-1)/(MA-ma+eps));
B=round((B-mb)*(L-1)/(MB-mb+eps));
n=zeros(L);
x=0:L-1;
for i=0:L-1
n(i+1,:) = histc(B(A==i),x,1);
end
end

is about 10x quicker, and gives the same result (assuming that the input is sensible, i believe you get different behavior if you pass in all nan's, or anything which would cause matlabs 'hist' function to error) ; given that the mi function calls hist on both the inputs individually first, this should not be a problem.

also there is need for a bugfix in mi, 256 bins are hard coded, irrespective of what you pass in.

the minf function also does unnecessary allocation.

Using the functions above and below rather than the attached the running time is about 22 seconds rather than 240, for my set of MI calculations. (2500 calls for MI between two 50x50 windows)

function I=fast_mi(A,B,L)
%MI Determines the mutual information of two images or signals
%
% I=mi(A,B)
%
% Assumption: 0*log(0)=0
%

% jfd, 15-11-2006
% 01-09-2009, added case of non-double images

A=double(A);
B=double(B);

na = fast_hist(A(:),L);
na = na/sum(na);

nb = fast_hist(B(:),L);
nb = nb/sum(nb);

n2 = fast_hist2(A,B,L);
n2 = n2/sum(n2(:));

I=sum(minf(n2,na'*nb));
%I=sum(I);

% -----------------------

function y=minf(pab,papb)

I=find(papb(:)>1e-12 & pab(:)>1e-12); % function support
y=pab(I).*log2(pab(I)./papb(I));

Comment only
01 Sep 2009 Jose Delpiano

### Jose Delpiano (view profile)

I solved problems related with data types. Now mi() accepts different types of images and signals (not only double images or signals). However, I am testing it with Matlab 7 (R14). Please leave a comment if it doesn't work with other versions.

Comment only
26 Feb 2009 Chaos

### Chaos (view profile)

Program bombs with 2007a

??? Error using ==> mtimes
Integers can only be combined with integers of the same class, or scalar
doubles.

Error in ==> hist at 73
xx = miny + binwidth*(0:x);

Error in ==> mi at 12
na = hist(A(:),256);

Comment only
06 May 2008 nicola rebagliati

the following codes raise errors:
"
A = uint8(zeros(4));
mi(A,A);
"

where uint8 is the usual type for images

"
A = zeros(4);
mi(A,A);
"

gives NaN.

10 Feb 2008 Todd Pataky

thank you for your well-written and clear code.

08 Jul 2007 jega xx

thank you very much for the code, really appreaciate it.

20 May 2007 liyakath unisa
07 Mar 2007 Lily Tang
01 Sep 2009 1.1

Added support for signals (1D matrices) and for non-double images.

24 Aug 2011 1.5

Speed improvements suggested in comments by Andrew Hill.

23 Feb 2015 1.6

Available as toolbox