## getKullbackLeibler(​P,Q)

Version 1.0.0.0 (2.94 KB) by
Compute Kullback-Leibler divergence

Updated 16 May 2017

call:

KLD = getKullbackLeibler(P,Q)

Compute Kullback-Leibler divergence of probability distribution Q from probability distribution P.
P represents the "true" distribution of data, observations, or a theoretical distribution.
whereas Q typically represents a theory, model, description, or approximation of P.
Both P and Q are thus distribution (obtained e.g. using 'getDensity.m') computed over the same range
(i.e. the same Bins -- if not, obviously it makes no sense to compare them).

The Kullback-Leibler divergence is an non-symmetric measure (see below) of the difference between
two probability distributions P and Q. Specifically, the Kullback–Leibler divergence of Q from P,
is a measure of the information lost when Q is used to approximate P. The Kullback–Leibler divergence
measures the expected number of extra bits (so intuitively it is non negative) required to code samples
from P when using a code optimized for Q, rather than using the true code optimized for P.
Although it is often intuited as a metric or distance, the Kullback–Leibler divergence is not a true
metric — for example, it is not symmetric: the Kullback–Leibler divergence from P to Q is generally
not the same as that from Q to P.

NOTE:
The code treat P and Q as DISCRETE probability distributions

----------------------------------------------------------------------------------------------------------------------------------------------------------

INPUT

P : Reference Probability distribution (an <M x 1> vector, where M is the no. of bins used to compute the distribution)
Q : Probability distribution I want to compare with the reference one (an <M x 1> vector too)

OUTPUT

KLD : Kullback-Leibler divergence (a scalar)

### Cite As

Ruggero G. Bettinardi (2023). getKullbackLeibler(P,Q) (https://www.mathworks.com/matlabcentral/fileexchange/62981-getkullbackleibler-p-q), MATLAB Central File Exchange. Retrieved .

##### MATLAB Release Compatibility
Created with R2009a
Compatible with any release
##### Platform Compatibility
Windows macOS Linux