ENTROPY

Compute the Shannon entropy of a set of variables.

ENTROPY(X,P) returns the (joint) entropy for the joint distribution corresponding to object matrix X and probability vector P. Each row of MxN matrix X is an N-dimensional object, and P is a length-M vector containing the corresponding probabilities. Thus, the probability of object X(i,:) is P(i).
ENTROPY(X), with no probability vector specified, will assume a uniform distribution across the objects in X.

If X contains duplicate rows, these are assumed to be occurances of the same object, and the corresponding probabilities are added. (This is actually the only reason that object matrix X is needed -- to detect and merge repeated objects. Of course, the entropy itself only depends on the probability vector P.) Matrix X need NOT be an exhaustive list of all *possible* objects in the universe; objects that do not appear in X are simply assumed to have zero probability.

The elements of probability vector P must sum to 1 +/- .00001.

See also: MUTUALINFO