Code covered by the BSD License  

Highlights from
Information Theory Toolbox


5.0 | 4 ratings Rate this file 195 Downloads (last 30 days) File Size: 3.95 KB File ID: #35625

Information Theory Toolbox



Functions for Information theory, such as entropy, mutual information, KL divergence, etc

| Watch this File

File Information

This toolbox contains functions for discrete random variables to compute following quantities:
2)Joint entropy
3)Conditional entropy
4)Relative entropy (KL divergence)
5)Mutual information
6)Normalized mutual information
7)Normalized variation information

This toolbox is a tweaked and bundled version of my previous submissions.
Note( previous single function submission will be removed soon).


Mutual Information and Normalized Mutual Information inspired this file.

MATLAB release MATLAB 7.13 (R2011b)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (10)
07 Oct 2014 Partha

I got different result using entropy(sig) and wentropy(sig,'shannon'). Can any one explain this?

19 Jul 2014 Subash Padmanaban

Change line 16:
Mx = sparse(idx, round(x), 1,n,k,n);

Apply the same changes to all sparse operations if the program throws the same error.

10 Dec 2013 Nejc Ilc

Very useful and efficient toolbox, thank you. However, there is a bug in the nmi.m. last sentence should read:
z = sqrt((MI/Hx)*(MI/Hy));
Output variable is "z" and not "v". But this is obvious a typo, so it does not influence my rating.

25 May 2013 shi

??? Error using ==> sparse
Index into matrix must be an integer.

Error in ==> mutualInformation at 16
Mx = sparse(idx,x,1,n,k,n);

is there anybody can help me?

27 Nov 2012 Maksim

Take back my last comment.

27 Nov 2012 Maksim  
27 Nov 2012 Maksim


returns 0.96


returns 0.91

Are you sure this is working correctly?

17 Sep 2012 Zulkifli Hidayat

In Windows 7 64-bit and using Matlab 2011b 64-bit, I got error for the following simple code:

x = randn(10,1);
y = randn(10,1);
z = mutualInformation(x,y)

Error message:

Error using sparse
Sparse matrix sizes must be non-negative integers less than MAXSIZE as defined by
COMPUTER. Use HELP COMPUTER for more details.

Error in mutualInformation (line 16)
Mx = sparse(idx,x,1,n,k,n);

Is there anybody tell me why?

12 May 2012 Jeff  
03 Apr 2012 ali Abusnina

Is there anyway how can we make these measure on time cries data?
can anyone help please

Contact us