|
David Heslop wrote:
> Hi All,
> I’m trying to use "normfit" to estimate the 95% confidence
> limit for the mean of a series of normally distributed
> experimental data points. Each point has an individual
> weight according to uncertainties in the experimental
> routine and I would like to take them into account in the
> calculation. From looking at the code of normfit I can see
> that the 95% confidence limit for the mean (muhat) is
> derived from the standard deviation (sigmahat) and the
> number of data points (m) in the following way:
>
> Lower_limit=muhat+tinv(0.025,m-1)*sqrt(sigmahat.^2/n);
> Upper_limit=muhat-tinv(0.025,m-1)*sqrt(sigmahat.^2/n);
>
> So (finally) here is my question. If I simply use a weighted
> mean and weighted standard deviation in the formulas above
> will that provide me with a correctly weighted 95%
> confidence interval, or will the weights play some role in
> also modifying the value of m?
Dave, first off, I think you are working with a somewhat old version of
NORMFIT. I can't remember if there was ever a version that advertised
that argument as weights, but it's important to point out that the
current NORMFIT advertises it as "frequencies", although it does not
enforce that interpretation. See, for example, the notes in the code
that follow the M help in the M file itself:
To compute weighted maximum likelihood estimates (WMLEs) for mu and
sigma, you can provide weights, normalized to sum to LENGTH(X), in FREQ
instead of frequencies. In this case, NORMFIT computes the WMLE for
mu. However, when there is no censoring, the estimate computed for
sigma is not exactly the WMLE. To compute the WMLE, multiply the value
returned in SIGMAHAT by (SUM(FREQ) - 1)/SUM(FREQ). This correction is
needed because NORMFIT normally computes SIGMAHAT using an unbiased
variance estimator when there is no censoring in the data. When there
is censoring, the correction is not needed, since NORMFIT does not use
the unbiased variance estimator in that case.
So I think the answer to your question is, no those CIs are not exactly
right, though it would be easy enough for you to modifying SIGMAHAT as
above and compute your own CIs. First, you more or less have to assume
that the weights represent something proportional to the inverse s.d.'s,
or even just a weighted estimate of sigma is probably not useful. But
if that's reasonable, then the sampling variance of the weighted
estimate for mu is sigma^2/n, and if you have a reasonable estimate of
sigma^2 (like, say, the weighted MLE), then you're in business.
Of course for large sample sizes, the modification will not do much.
But in the bigger picture, I think all of that assumes that you're
conditioning on the weights, and you'll have to decide if the resulting
CIs make any sense. I don't know if there is any accepted practice for
weighted CIs. The math is simple; you'll want to work this all out to
make sure you are doing what you think you're doing.
Hope this helps.
- Peter Perkins
The MathWorks, Inc.
|
Feed for this Thread