Information Theory Toolbox v1.0

Implementation of major algorithms related to information theory for categorical data including a si

Description

December 13th, 2007. Information Theory Toolbox v1.0

A complete toolbox including the implementation of several algorithms related to information theory for categorical data. The methods included are:
 -entropy (entropy.m)
 -conditional entropy (condentropy.m)
 -mutual information (mutualinformation.m)
 -redundancy (redundancy.m)
 -symmetric uncertainty
 -Kullback-Leibler divergence (kldivergence.m)
-Jensen-Shannon divergence (klsymdivergene.m)

Mutual information implementation includes a permutation test to estimate significance based on Monte Carlo sampling.

It is a cross-platform implementation with no libraries or external code. It only contains Matlab code.

Type help for each function to get a detailed description and examples.

It only includes Matlab code,

Citation:

If you use them for your academic research work,please kindly cite this
toolbox as:
Joaquin Goñi, Iñigo Martincorena. Information Theory Toolbox v1.0.
University of Navarra - Dpt. of Physics and Applied Mathematics &
Centre for Applied Medical Research. Pamplona (Spain).

• MATLAB release: MATLAB 7 (R14)
• Other requirements: No particular requirements or toolboxes needed.

[xunkai wei]

Hi, why not use vector form ? You can further speed them up using vector form. Keep polishing.

[Eric Li]

I want to compute Conditional Mutual Information , can you help me ,it seems that the toolbox does not contain this algorithm.

[Ashwin Sundar]

Nice product!!!!!!!!!!!!!!!!!

[Do Quoc Bao]

In the "function kl = kldivergence (x,y)", there is a serious error !
Look at the 62 and 63 line:
ipOcurrences = numel(find(x==valuesX(i))); %computes i-ocurrences at x
iqOcurrences = numel(find(y==valuesY(i))); %computes i-ocurrences at y
Note that valuesX and valuesY are distinct values for vector x and vector y, respectively. But valuesX(i) can be different from valuesY(i). Therefore pi and qi are not probability of the same index !!!!!!

[Yuval Aviel]

All functions here takes random variables as input, not probability distribution, as commonly defined.
This is very confusing.

In addition, these function will probably work on categorical data only.