Total Kullback-Leibler (tKL) divergence between multivariate normal probability density functions.
16 Jun 2011
(Updated 28 Feb 2012)
tKL between two multivariate normal probability density functions.
function delta = tKLFunc(p,q)
%% total Kullback Leibler divergence (tKL) delta_f(p,q) between two
%% multivariate normal probabiltiy density functions.
% f is the convex and differentiable generator function
% f is the negative entropy;
% p: Gaussian distribution with mean 0 and variance sigma;
% q: Gaussian distribution with mean 0 and variance sigma;
% delta: the tKL between p and q.
% Author: Meizhu Liu
% E-Mail: firstname.lastname@example.org
% Date : December 28 2010
% Baba C. Vemuri, Meizhu Liu, Shun-Ichi Amari and Frank Nielsen,
% Total Bregman Divergence and its Applications to DTI Analysis,
% IEEE Transactions on Medical Imaging (TMI'10), 2010.
% Meizhu Liu, Baba C. Vemuri, Shun-Ichi Amari and Frank Nielsen,
% Total Bregman Divergence and its Applications to Shape Retrieval,
% IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10),
%% Example Usage:
% p.sigma = [1 0;0 1];
% q.sigma = [2 0;0 2];;
% delta = tKLFunc(p,q);
%% The main part
P = p.sigma;
Q = q.sigma;
invP = inv(P);
invQ = inv(Q);
d = size(Q,1); %the dimension of the dataset following this distribution
c = 0.75*d+d^2*log(2*pi)/2+(d*log(2*pi))^2/4;
delta = (log(det(invP*Q))+trace(invQ*P)-d);
tmp = c+log(det(Q))^2/4-d*(1+log(2*pi))/2*log(det(Q));
delta = delta/2/sqrt(tmp);