File Exchange

image thumbnail

Mutual Information

version 1.2.0.0 (1.33 KB) by Will Dwinnell
Calculates the mutual information between two discrete variables (or a group and a single variable).

53 Downloads

Updated 03 Jan 2012

View License

MutualInformation: returns mutual information (in bits) of discrete variables 'X' and 'Y'

I = MutualInformation(X,Y);

I = calculated mutual information (in bits)
X = variable(s) to be analyzed (column vector)
Y = variable to be analyzed (column vector)

Note 1: Multiple variables may be handled jointly as columns in
matrix 'X'.
Note 2: Requires the 'Entropy' and 'JointEntropy' functions.

Comments and Ratings (13)

Hanqing Ma

I think there is something wrong! Clearly wrong results!
a=rand(1,10000);
b=rand(1,10000);
I=MutualInformation(a',b')

I =

13.2877

>> I=MutualInformation(a',a')

I =

13.2877

please someone email me i need help, i want 'Mutual information' and 'Information Gain' full code and also how to access the Datasets. please my email is mibrar868@gmail.com

Can this code generate a mutual information matrix for each feature column to every other feature column?

Duo Hao

good job

Batuhan

Those who cannot find Entropy.m and JointEntropy.m files should search the Matlab File Exchange. These files ara available there, by the same author.

Debanjan

where can I find the 'Entropy' and 'JointEntropy' functions? The Mutual Information does not seem to run without these functions

By "discrete", I mean integer values (intended to act as codes for distinct symbols).

This routine is intended for use with discrete variables, not continuous ones.

Jun

I am not sure if I run the code right or there is a bug in the code.

I ran the following and got the answers below. Does not seem right to me. A and B are completely independent random numbers, which I verified.

>> A=rand(1,100);
>> B=rand(1,100);
>> MutualInformation(A',A')

ans =

6.6439

>> MutualInformation(B',B')

ans =

6.6439

>> MutualInformation(A',B')

ans =

6.6439

Good Job

Updates

1.2.0.0

I have updated the description to clarify the use of this routine.

MATLAB Release Compatibility
Created with R12
Compatible with any release
Platform Compatibility
Windows macOS Linux

Discover Live Editor

Create scripts with code, output, and formatted text in a single executable document.


Learn About Live Editor