Thread Subject: Probability density function (pdf) gave me

When I used the pdf of the multivariate normal distribution to
calculate the probability of some vector X in Matlab, I got
probabilities far greater than 1 (like, hundreds). This is the
formula I used:

p = exp(-1/2*(X-mu)'*cov^(-1)*(X-mu))/(sqrt((2*pi)^n)*det(cov))

cov: covariance matrix
mu: mean
det: determinant
n: dimension of the vector, which is also the number of the variables


If p donates the probability, it should not be greater than one,
should it?

Can someone plese give me some hint? Thank you so much!

Jessie

Jessie skrev:
> When I used the pdf of the multivariate normal distribution to
> calculate the probability of some vector X in Matlab, I got
> probabilities far greater than 1 (like, hundreds). This is the
> formula I used:
>
> p = exp(-1/2*(X-mu)'*cov^(-1)*(X-mu))/(sqrt((2*pi)^n)*det(cov))
>
> cov: covariance matrix
> mu: mean
> det: determinant
> n: dimension of the vector, which is also the number of the variables
>
>
> If p donates the probability, it should not be greater than one,
> should it?
>
> Can someone plese give me some hint? Thank you so much!

Assuming the formula is correct and there are no typos in
your program -- as I am sure you have checked -- I would
start by having a closer look at the covariance matrix.

There are basically two potential problems here, normalization
and bias.

I would assume the covariance matrix ought to be normalized
so that the values on the diagonal are exactly unity. Have you
made sure they are?

The second potential problem, is bias. I have never studied
formal statistsics, so I do not know for cetrtain, but I
would not be surprised if there is some restriction on the
determinant of the covariance matrix, like

det(cov) <= 1 [1]

or something like that. Now, if such a restriction holds,
it would be valid fro the *true* covariance matrix.
Depending on the application, you might have an estimated
covariance matrix. If the estimator introduces a bias, the
condition [1] -- if such a thing exists -- might be violated.

Just my 2c.

Rune

On Fri, 19 Aug 2005 17:22:06 -0400, Jessie <jess.shen@gmail.com>
wrote:

>When I used the pdf of the multivariate normal distribution to
>calculate the probability of some vector X in Matlab, I got
>probabilities far greater than 1 (like, hundreds). This is the
>formula I used:
>
>p = exp(-1/2*(X-mu)'*cov^(-1)*(X-mu))/(sqrt((2*pi)^n)*det(cov))
>
>cov: covariance matrix
>mu: mean
>det: determinant
>n: dimension of the vector, which is also the number of the variables
>
>
>If p donates the probability, it should not be greater than one,
>should it?

No. p(x) is the pdf, its integral is a probability.

Try to plot your p(x) with n=1 and cov=0.1

 (n=1, than the covariance matrix is only a number)

p(x=0) is near to 4

integral from -inf to +inf is 1, as expected.

See, for example,
http://en.wikipedia.org/wiki/Normal_distribution
or
http://mathworld.wolfram.com/NormalDistribution.html



>n: dimension of the vector, which is also the number of the variables

Be careful! if n is the dimension of your 'space', and n is also the
number of the variables that you use in order to evaluate the
covariance matrix, the covariance matrix will be wrong!

The covariance matrix will be a nxn symmetric matrix, so it has
m=(n^2-n)/2 different elements. You cannot determine m elements form
n<m elements!
[ maybe I could misandestood you and this is not of your interess]


Sorry for the poor english and hth









>
>Can someone plese give me some hint? Thank you so much!
>
>Jessie

Thanks for your message. I didn't normalize the covariance matrix.
Maybe that could have caused the problem. I need to work on that and
see how it goes. Have a great weekend! lol

Thanks for the reply. You are right. p is not the probability. It's
the pdf. My math has failed me again. =)

Jess
 
> On Fri, 19 Aug 2005 17:22:06 -0400, Jessie
<jess.shen@gmail.com>
> wrote:
>
>>When I used the pdf of the multivariate normal distribution to
>>calculate the probability of some vector X in Matlab, I got
>>probabilities far greater than 1 (like, hundreds). This is the
>>formula I used:
>>
>>p = exp(-1/2*(X-mu)'*cov^(-1)*(X-mu))/(sqrt((2*pi)^n)*det(cov))
>>
>>cov: covariance matrix
>>mu: mean
>>det: determinant
>>n: dimension of the vector, which is also the number of the
> variables
>>
>>
>>If p donates the probability, it should not be greater than one,
>>should it?
>
> No. p(x) is the pdf, its integral is a probability.
>
> Try to plot your p(x) with n=1 and cov=0.1
>
> (n=1, than the covariance matrix is only a number)
>
> p(x=0) is near to 4
>
> integral from -inf to +inf is 1, as expected.
>
> See, for example,
> <http://en.wikipedia.org/wiki/Normal_distribution>
> or
> <http://mathworld.wolfram.com/NormalDistribution.html>
>
>
>
>>n: dimension of the vector, which is also the number of the
> variables
>
> Be careful! if n is the dimension of your 'space', and n is also
> the
> number of the variables that you use in order to evaluate the
> covariance matrix, the covariance matrix will be wrong!
>
> The covariance matrix will be a nxn symmetric matrix, so it has
> m=(n^2-n)/2 different elements. You cannot determine m elements
> form
> n<m elements!
> [ maybe I could misandestood you and this is not of your interess]
>
>
> Sorry for the poor english and hth
>
>
>
>
>
>
>
>
>
>>
>>Can someone plese give me some hint? Thank you so much!
>>
>>Jessie
>
>