Thread Subject:

standard deviation from range

I have some statistics from a set of univariate data including, number of observations, mean and range. Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.

Anybody know how to do this or a good source?

thanks

Mike
I don't know what you mean by the "distribution of the standard deviation." Wouldn't your population have just ONE standard deviation, which you can get with the std() function? How are you having more than one standard deviation? Do you not have the original individual observations and instead have just the stats from some number of *sets* of observations?

"Image Analyst" wrote in message <j23h61$cbr$1@newscl01ah.mathworks.com>...
>
> Do you not have the original individual observations and instead have just the stats from some number of *sets* of observations?
===========================

If so, use HIST or HISTC to get the distribution of the statistics.

On 8/12/2011 10:34 AM, Image Analyst wrote:
> Mike
> I don't know what you mean by the "distribution of the standard
> deviation." Wouldn't your population have just ONE standard deviation,
> which you can get with the std() function? How are you having more than
> one standard deviation? Do you not have the original individual
> observations and instead have just the stats from some number of *sets*
> of observations?

I read it as having only one set of summary statistics from a presumed
underlying distribution but the question (and what data are available)
of what, specifically, is desired to estimate is so unclear it doesn't
seem worthwhile to worry too much until OP clarifies.

There is, of course, in the worst case, the Tchebyshev (sp?; I can never
decide on a consistent variant :) ) inequality that is conservative (in
some senses) if that will do...

--

Yes, I'm confused about the "distribution of the standard deviation"
statement also. If you are looking for the standard deviation of the
sample, then here's a solution. If your range is the min and max
values, then you can assign probabilities to these based on your
sample size. For instance, the min value will have a probability of 1/
(N+1), and your max value will have a probability of N/(N+1). If your
distribution is normal, than values of these away from the 0.5 value
should be rather close (meaning, 0.5 - 1/(N+1) should be close to N/(N
+1) - 0.5). You can then use the MATLAB norminv function to determine
the sigma level for the probability.

>> N = 100;
>> x_bar = 0; %your mean value
>> range = [-10 10];
>> prob_low = 1/(N+1);
>> prob_high = N/(N+1);
>> sig_level = norminv(prob_high,0,1);
>> sig_val = (range(2)-x_bar)/sig_level;

sig_val should give you value of the standard deviation. Of course, if
your data isn't normal, or if there are outliers, then the above
method doesn't work.

hth,
Frank

"Mike " <mike.curran@rbccm.com> wrote in message <j23b6t$k6n$1@newscl01ah.mathworks.com>...
>
> I have some statistics from a set of univariate data including, number of observations, mean and range. Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.
>
> Anybody know how to do this or a good source?
>
> thanks
- - - - - - - - - -
  If you are dealing with a population with a known mean and standard deviation, it should be possible to compute a conditional distribution of the sample standard deviation given the number of observations and the range in the sample. However this answer would of course be heavily dependent on that original underlying known population mean and standard deviation. Without at least some a priori knowledge of such population statistics I am not sure you have asked a meaningful question. Perhaps you should explain in greater detail what you are asking.

  There is a very different question that could be asked for a given observed range and number in the sample. What mean and standard deviation on a normal population would give the maximum probability for this observation - in other words which gives the maximum likelihood? However that is not the same as asking for a distribution.

Roger Stafford

On Aug 13, 1:52 am, "Mike " <mike.cur...@rbccm.com> wrote:
> I have some statistics from a set of univariate data including, number of observations, mean and range.  Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.
>
> Anybody know how to do this or a good source?
>
> thanks

If by "range", you mean the difference between maximum and minimum,
then you can use the Universal threshold of Donoho & Johnstone, which
states that the expected maximum of N independent, identically
distributed, standard normal random samples is:
lambda=sqrt(2*log(N))*sigma
where sigma is an estimate of the standard deviation.
Inverting this and inserting range/2 for lambda will give you an
estimate of the SD.

On Aug 12, 9:52 am, "Mike " <mike.cur...@rbccm.com> wrote:
> I have some statistics from a set of univariate data including, number of observations, mean and range.  Assuming that the data come from a normal >population, I'd like the distribution of the standard deviation of the population.

There is only one population standard deviation.

I assume you want a single sample estimate of the
standard deviation, NOT the distribution of single
sample estimates frorn samples of size N.

If that is correct, I believe there are formulae from
the theory of "extreme value statistics" and/or
"order statistics" relating extreme values, mean
and variance.

> Anybody know how to do this or a good source?

I would search using the keywords "extreme value
statistics" and/or "order statistics"

Also, correct the ambiguity of your question and
post to sci.stat.math.

Hope this helps.

Greg

"Image Analyst" wrote in message <j23h61$cbr$1@newscl01ah.mathworks.com>...
> Mike
> I don't know what you mean by the "distribution of the standard deviation." Wouldn't your population have just ONE standard deviation, which you can get with the std() function? How are you having more than one standard deviation? Do you not have the original individual observations and instead have just the stats from some number of *sets* of observations?

Certainly, the population has one standard deviation, but as it is unknown, we can speak of its distribution based on the known info.

TideMan <mulgor@gmail.com> wrote in message <b962ded7-9cc6-437d-87e5-484e6ac8cc39@s18g2000prc.googlegroups.com>...
> On Aug 13, 1:52 am, "Mike " <mike.cur...@rbccm.com> wrote:
> > I have some statistics from a set of univariate data including, number of observations, mean and range.  Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.
> >
> > Anybody know how to do this or a good source?
> >
> > thanks
>
> If by "range", you mean the difference between maximum and minimum,
> then you can use the Universal threshold of Donoho & Johnstone, which
> states that the expected maximum of N independent, identically
> distributed, standard normal random samples is:
> lambda=sqrt(2*log(N))*sigma
> where sigma is an estimate of the standard deviation.
> Inverting this and inserting range/2 for lambda will give you an
> estimate of the SD.


Thanks TideMan - that looks like what I need

TideMan <mulgor@gmail.com> wrote in message <b962ded7-9cc6-437d-87e5-484e6ac8cc39@s18g2000prc.googlegroups.com>...
> On Aug 13, 1:52 am, "Mike " <mike.cur...@rbccm.com> wrote:
> > I have some statistics from a set of univariate data including, number of observations, mean and range.  Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.
> >
> > Anybody know how to do this or a good source?
> >
> > thanks
>
> If by "range", you mean the difference between maximum and minimum,
> then you can use the Universal threshold of Donoho & Johnstone, which
> states that the expected maximum of N independent, identically
> distributed, standard normal random samples is:
> lambda=sqrt(2*log(N))*sigma
> where sigma is an estimate of the standard deviation.
> Inverting this and inserting range/2 for lambda will give you an
> estimate of the SD.

I tried running
x=randn(N,1);

term = sqrt(2*log(N));

sdEst = (max(x)-min(x))/ (2*term)% seems low

sdEst1 = max(x)/term
sdEst2 = -min(x)/term

and keep getting estimates 5-10% low (ie .90-.95 instead of 1.0). Is there another factor in the formula, or am I doing something wrong??
thanks

On Aug 16, 6:57 am, "Mike " <mike.cur...@rbccm.com> wrote:
> TideMan <mul...@gmail.com> wrote in message <b962ded7-9cc6-437d-87e5-484e6ac8c...@s18g2000prc.googlegroups.com>...
> > On Aug 13, 1:52 am, "Mike " <mike.cur...@rbccm.com> wrote:
> > > I have some statistics from a set of univariate data including, number of observations, mean and range.  Assuming that the data come from a normal population, I'd like the distribution of the standard deviation of the population.
>
> > > Anybody know how to do this or a good source?
>
> > > thanks
>
> > If by "range", you mean the difference between maximum and minimum,
> > then you can use the Universal threshold of Donoho & Johnstone, which
> > states that the expected maximum of N independent, identically
> > distributed, standard normal random samples is:
> > lambda=sqrt(2*log(N))*sigma
> > where sigma is an estimate of the standard deviation.
> > Inverting this and inserting range/2 for lambda will give you an
> > estimate of the SD.
>
> I tried running
> x=randn(N,1);
>
> term = sqrt(2*log(N));
>
> sdEst = (max(x)-min(x))/ (2*term)% seems low
>
> sdEst1 = max(x)/term
> sdEst2 = -min(x)/term
>
> and keep getting estimates 5-10% low (ie .90-.95 instead of 1.0).  Is there another factor in the formula, or am I doing something wrong??
> thanks

No, that's about right.
I've found the same, even for very large N.