# Implement the "total variation distance" (TVD) in Matlab

42 views (last 30 days)
Sim on 3 Jul 2023
Edited: Bruno Luong on 4 Aug 2023
I am trying to implement the Total variation distance of probability measures (TVD) in Matlab.
Would it be correct to use the max function, in order to calculate the "supremum" of the TVD equation (here below)?
My attempt:
% Input
A =[ 0.444643925792938 0.258402203856749
0.224416517055655 0.309641873278237
0.0730101735487732 0.148209366391185
0.0825852782764812 0.0848484848484849
0.0867743865948534 0.0727272727272727
0.0550568521843208 0.0440771349862259
0.00718132854578097 0.0121212121212121
0.00418910831837223 0.0336088154269972
0.00478755236385398 0.0269972451790634
0.00359066427289048 0.00110192837465565
0.00538599640933573 0.00220385674931129
0.000598444045481747 0
0.00299222022740874 0.00165289256198347
0 0
0.00119688809096349 0.000550964187327824
0 0.000550964187327824
0.00119688809096349 0.000550964187327824
0 0.000550964187327824
0 0.000550964187327824
0.000598444045481747 0
0.000598444045481747 0
0 0
0 0.000550964187327824
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0.000550964187327824
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0.00119688809096349 0.000550964187327824];
P = A(:,1);
Q = A(:,2);
% Total variation distance (of probability measures)
d = max(abs(P-Q))
d = 0.1862

Bruno Luong on 4 Aug 2023
Edited: Bruno Luong on 4 Aug 2023
Supremum is very often implemented by max, since one can only list or compute a finite set on computer.
However your formula d = max(abs(P-Q)) is not correct to compute TVD.
According to this wiki page; correct formula is given bellow "When Ω is countable"
d = 0.5 * norm(P-Q,1)
or
d = 0.5 * sum(abs(P-Q));
Bruno Luong on 4 Aug 2023
Edited: Bruno Luong on 4 Aug 2023
Don't use the brute force implementation of the initial definition for any discrete pdf with more than 20 values (n = cardinal of Omega), rather use
dFormula = 0.5 * norm(P-Q,1)
The for-loop I made is just to illustrate the correctness of the formula. Just like no-one would computes the determinant of matrix 30 x 30 using Leibniz formula.
Sim on 4 Aug 2023
Ah ok..great..!! Many many thanks!
Then, I will use:
dFormula = 0.5 * norm(P-Q,1)

Hi Sim,
Upon searching, I found the exact question being asked on stackoverflow (I'm assuming it was posted by you only), where somebody has already answered the question. I am attaching the link to that answer for future reference:
Sim on 4 Aug 2023
Edited: Sim on 4 Aug 2023
Yes exactly! :-)