Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Old bug in corrcoef not yet fixed
Date: Thu, 8 Jan 2009 07:28:06 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 29
Message-ID: <gk49u6$ajh$1@fred.mathworks.com>
References: <gk3k0q$hko$1@fred.mathworks.com> <gk3oon$be3$1@fred.mathworks.com> <gk3so5$h1e$1@fred.mathworks.com>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1231399686 10865 172.30.248.37 (8 Jan 2009 07:28:06 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Thu, 8 Jan 2009 07:28:06 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1187260
Xref: news.mathworks.com comp.soft-sys.matlab:510371

"Pasco Alquim" <pasquimm@yahoo.com> wrote in message <gk3so5$h1e$1@fred.mathworks.com>...
> Sorry, but what I see in the docs is
> 
> [...]=corrcoef(...,'param1',val1,'param2',val2,...) specifies additional parameters and their values. Valid parameters are the following. 
>   'alpha' A number between 0 and 1 to specify a confidence level of 100*(1 - alpha)%. Default is 0.05 for 95% confidence intervals.
> 
> No mention that, when 'alpha' is provided, one must have 4 argouts
> 
> And why should p be the probability for the 95% confidence only? Which is in fact is what it does.
> 
> We can confirm that by hacking the code and changing the value of variable rv before it calls tpvalue. We can than use the values from the table at 
> http://physics.mercer.edu/Younce/pearson.html
> to confirm that what is return is the 95% confidence value.
> 
> Why the preference to 95%?
> Why not let the user choose the prefered confodence level, as the documentation suggests?
--------------
  I strongly disagree with what you are saying about the documentation, Pasco.  Where it describes the 'p' output, there is no mention of any confidence level except the indirect reference, "If P(i,j) is small, say less than 0.05, then the correlation R(i,j) is significant," and this is simply a conclusion that is left for the user to draw and not a statement that p depends upon a confidence level input.  They state further that "The p-value is computed by transforming the correlation to create a t statistic having n-2 degrees of freedom, where n is the number of rows of X."  There is nothing in such a computation that involves any notion of a confidence level.  Your statement "why should p be the probability for the 95% confidence only?" is just plain wrong; 'p' has nothing to do with a confidence level, 95% or otherwise.

  It is only when the specific confidence bounds outputs 'rlo' and 'rup' are called for that a confidence level is needed.  If you don't specify 'alpha', they assume it is .05 for 95% confidence as default.  I refer you to a couple of websites below for an explanation of the procedure involved in deducing these confidence bounds from each correlation value and, given the 'alpha' level, the appropriate values for 'rlo' and 'rup'.  This is the only place the 'alpha' quantity input is used.  The 'p' quantities are not affected by its setting.  You can do your own independent calculation using the Fisher transform and its inverse to see if Mathworks does it right.  The web sites are located at:

http://en.wikipedia.org/wiki/Fisher_transformation

http://www.stat.psu.edu/online/development/stat505/06_propmean/06_propmean_print.html

I am sure there are a great many more sites that give a good explanation of this procedure.  Use a Google search for something like "Fisher Transform" and "confidence levels".

Roger Stafford