Compute Kullback-Leibler divergence


Updated 16 May 2017

View License


KLD = getKullbackLeibler(P,Q)

Compute Kullback-Leibler divergence of probability distribution Q from probability distribution P.
P represents the "true" distribution of data, observations, or a theoretical distribution.
whereas Q typically represents a theory, model, description, or approximation of P.
Both P and Q are thus distribution (obtained e.g. using 'getDensity.m') computed over the same range
(i.e. the same Bins -- if not, obviously it makes no sense to compare them).

The Kullback-Leibler divergence is an non-symmetric measure (see below) of the difference between
two probability distributions P and Q. Specifically, the Kullback–Leibler divergence of Q from P,
is a measure of the information lost when Q is used to approximate P. The Kullback–Leibler divergence
measures the expected number of extra bits (so intuitively it is non negative) required to code samples
from P when using a code optimized for Q, rather than using the true code optimized for P.
Although it is often intuited as a metric or distance, the Kullback–Leibler divergence is not a true
metric — for example, it is not symmetric: the Kullback–Leibler divergence from P to Q is generally
not the same as that from Q to P.

The code treat P and Q as DISCRETE probability distributions

Description inspired by Wikipedia page:


P : Reference Probability distribution (an <M x 1> vector, where M is the no. of bins used to compute the distribution)
Q : Probability distribution I want to compare with the reference one (an <M x 1> vector too)


KLD : Kullback-Leibler divergence (a scalar)

Cite As

Ruggero G. Bettinardi (2023). getKullbackLeibler(P,Q) (, MATLAB Central File Exchange. Retrieved .

MATLAB Release Compatibility
Created with R2009a
Compatible with any release
Platform Compatibility
Windows macOS Linux
Tags Add Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!
Version Published Release Notes