Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Expectation Value or Most Probable Value???

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 27 Feb, 2012 00:49:11

Message: 1 of 14

Dear Matlab Community,

This is a question that has bothered me for a long time.

Let's say I have a model...
y = f(t) = A*exp(-B*t) + C;
where, A is an amplitude, B is a rate parameter and C is a constant.

Let's say that an experiment collects samples of the signal y up to time t << infiniti.

I would argue that if samples are collected up to time t << infiniti, then the constant C should not be estimated by the model using nonlinear least squares fitting because it is only at t = infiniti, that the first term goes to zero and can give you y = C.

So, let's say that the model is not used to find C. However, let's say that there is information in the data that lets you estimate C from a probability density function using maximum likelihood.

Let's say that the PDF is not gaussian, rather some other PDF where the mean (expected value or 1st moment) does NOT equal the mode (the most probable value or the value with highest density).

What distribution descriptor would you use to substitute for the value of C? Would you use the most probable value or the expected value?

Thanks,

Eric

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 27 Feb, 2012 19:19:11

Message: 2 of 14

bump

Subject: Expectation Value or Most Probable Value???

From: John D'Errico

Date: 27 Feb, 2012 21:20:14

Message: 3 of 14

"Eric Diaz" wrote in message <jieju7$1be$1@newscl01ah.mathworks.com>...
> Dear Matlab Community,
>
> This is a question that has bothered me for a long time.
>
> Let's say I have a model...
> y = f(t) = A*exp(-B*t) + C;
> where, A is an amplitude, B is a rate parameter and C is a constant.
>
> Let's say that an experiment collects samples of the signal y up to time t << infiniti.
>
> I would argue that if samples are collected up to time t << infiniti, then the constant C should not be estimated by the model using nonlinear least squares fitting because it is only at t = infiniti, that the first term goes to zero and can give you y = C.
>
> So, let's say that the model is not used to find C. However, let's say that there is information in the data that lets you estimate C from a probability density function using maximum likelihood.
>
> Let's say that the PDF is not gaussian, rather some other PDF where the mean (expected value or 1st moment) does NOT equal the mode (the most probable value or the value with highest density).
>
> What distribution descriptor would you use to substitute for the value of C? Would you use the most probable value or the expected value?
>

Why are you asking this on a matlab site, instead of a
more appropriate place? Perhaps someplace where they
do statistics? The fact that you have gotten no response
should be a hint.

John

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 27 Feb, 2012 23:23:11

Message: 4 of 14

Possibly because this is one of the only forums that I really know about. Do you have any suggestions for a proper statistical forum to go to? or would you rather just be critical without providing any actually useful information.

Subject: Expectation Value or Most Probable Value???

From: John D'Errico

Date: 28 Feb, 2012 04:55:13

Message: 5 of 14

"Eric Diaz" wrote in message <jih38v$e6r$1@newscl01ah.mathworks.com>...
> Possibly because this is one of the only forums that I really know about. Do you have any suggestions for a proper statistical forum to go to? or would you rather just be critical without providing any actually useful information.

Um, at least I was willing to point out that you were not
looking in the right place. All you bothered to do was to
ask, and then try bumping it up to the top when nobody
showed any interest. I was actually the MOST helpful
person here in that respect.

There are a lot of places to look, if you had bothered to
do so. They are not even hard to find, so you did not
bother.

Start with Google groups. Since this IS a USENET group
you are posting on, why not look there? There are several
statistics groups there as I recall.

Or stackexchange, but NOT stackoverflow. Had you looked,
you would find something there. I'll bet I could find a few
others easily, or rather, Google would do it for me.

John

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 28 Feb, 2012 08:42:13

Message: 6 of 14

Looking around on google to find a quality statistics group is not as easy as you imply. I would much rather have a recommendation from someone who has actually been on one of these forums and found it to be useful. So, while you have suggested a couple sites, if you have never been to them yourself, how do you know they are any good? It would seem that if you are recommending some random site for help, you are not being that helpful, again as usual.

Subject: Expectation Value or Most Probable Value???

From: John D'Errico

Date: 28 Feb, 2012 11:10:11

Message: 7 of 14

"Eric Diaz" wrote in message <jii415$h2g$1@newscl01ah.mathworks.com>...
> Looking around on google to find a quality statistics group is not as easy as you imply.

Then you need to start learning how to use the internet.
But even if not, you did not take the several suggestions
I gave you, as I told you explicitly where to find those
groups.

> I would much rather have a recommendation from someone who has actually been on one of these forums and found it to be useful. So, while you have suggested a couple sites, if you have never been to them yourself, how do you know they are any good? It would seem that if you are recommending some random site for help, you are not being that helpful, again as usual.

And you continue to insult the one person who is willing
to be helpful at all here, while still not bothering to make
any effort at all on your part.

You post a question that is not even remotely about
matlab on a site that explicitly answers matlab questions.
I have given you several places that would be significantly
better in terms of their fit for this question. Which one is
best?

The fact that statisticians will be the ones reading your
question will considerably improve the odds that it will
get an answer though. The statistics usenet groups you
will find on google have been around as long as c.s-s.m
(the matlab usenet group) has been in existence. You
could start there, which, by the way, is the first place I
told you to look. Since you cannot bother to make ANY
effort at all, a place to look might be:

sci.stat.math

although you may find the newer group on stackexchange
has heavier volume.

plonk (but probably have no idea what that means anyway.)

Subject: Expectation Value or Most Probable Value???

From: Tom Lane

Date: 28 Feb, 2012 15:33:39

Message: 8 of 14

> y = f(t) = A*exp(-B*t) + C;
> where, A is an amplitude, B is a rate parameter and C is a constant.
>
> Let's say that an experiment collects samples of the signal y up to time t
> << infiniti.
>
> I would argue that if samples are collected up to time t << infiniti, then
> the constant C should not be estimated by the model using nonlinear least
> squares fitting because it is only at t = infiniti, that the first term
> goes to zero and can give you y = C.

I missed this original question but saw the debate about whether it is a
MATLAB question. Let me give a MATLAB answer.

There can be plenty of information about C even if we never reach t=Inf.
Similarly, we might fit a model like a+b*x and estimate the intercept a,
even if we have data where x is not close to zero. Here's an example where I
can estimate the parameters reasonably well. Of course, there will be other
datasets where the estimates are not good; I don't know how things would go
with your data.

>> t = linspace(0,1)';
>> A = 1; B = 2; C = 3;
>> y = A*exp(-B*t) + C + randn(size(t))/10;
>> [b,~,~,covb] = nlinfit(t,y,@(abc,x) abc(1)*exp(-abc(2)*t)+abc(3),[1 1 1])
b =
    0.9809 2.0741 3.0204
covb =
    0.0025 -0.0107 -0.0025
   -0.0107 0.0946 0.0177
   -0.0025 0.0177 0.0036

Then you had a question about a distribution descriptor for C, but I did not
follow that. For example, suppose if you decided C had some sort of Weibull
distribution. I don't understand where you would go with that, and how you
would use that information to estimate A and B. But I would consider neither
the mode nor the mean, but perhaps the median (50% point) if I had to
describe C using one numeric value.

-- Tom

Subject: Expectation Value or Most Probable Value???

From: Steven_Lord

Date: 28 Feb, 2012 18:17:12

Message: 9 of 14



"Tom Lane" <tlane@mathworks.com> wrote in message
news:jiis4j$3l7$1@newscl01ah.mathworks.com...
>> y = f(t) = A*exp(-B*t) + C;
>> where, A is an amplitude, B is a rate parameter and C is a constant.
>>
>> Let's say that an experiment collects samples of the signal y up to time
>> t << infiniti.
>>
>> I would argue that if samples are collected up to time t << infiniti,
>> then the constant C should not be estimated by the model using nonlinear
>> least squares fitting because it is only at t = infiniti, that the first
>> term goes to zero and can give you y = C.
>
> I missed this original question but saw the debate about whether it is a
> MATLAB question. Let me give a MATLAB answer.
>
> There can be plenty of information about C even if we never reach t=Inf.

In addition to what Tom said, even if in infinite precision arithmetic
A*exp(-B*t) will never be exactly 0 you may have data that makes it
numerically 0 in double precision.

y = exp(-750) % underflows to exactly 0

Basically if B*t is significantly greater than -log(eps(0)) that calculation
will underflow. Depending on the scale of your B, a modest t may cause
underflow in the first term. Even if it doesn't directly underflow, unless A
is much, much larger than C the miniscule contribution of A*exp(-B*t) may be
swamped by C's contribution.

y = exp(-500) % Small enough to be considered zero for many purposes

If you want a more statistics-based answer rather than a MATLAB-based
answer, you're probably going to want to go somewhere focused on statistics,
like the sci.stat.math newsgroup (accessible via Google Groups,
http://groups.google.com, if your news server doesn't carry it.)

*snip*

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 28 Feb, 2012 22:53:13

Message: 10 of 14

You're right I don't know what 'plunk' is. Nor do I know what a 'USENET' is. Nor did I know that the MATLAB newsreader was a 'USENET'. I have just been using the MATLAB forum that is available through MATLAB central.

Apparently, you think that knowing these things is common knowledge, which shows what a totally isolated, passive-agressive and probably schizoid person you are. And I'm actually pretty happy that I don't know what 'plunk' is, because if I did, I would probably be as useless and condescending as you.

I don't see how discussing a math topic such as parameter estimation of a constant in a nonlinear model is not relevant on a mathworks forum.

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 28 Feb, 2012 23:17:14

Message: 11 of 14

Thanks Steven.

In the problem set I am applying this to, the rate with which the signal intensity can decay varies significantly.

To give you an idea,
time is in the set (0,...~100 milliseconds)
decay rates are on the order of 1/2000 msec^-1 (very slow decay) all the way to 2 msec^-1 (very fast decay)
The bulk though, is typically around 1/1 to 1/200

I don't think with these rates there is typically the problem of 'underflow' as you brought up. The value of the signal typically never even gets close to going all the way to zero during the sampling window.

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 28 Feb, 2012 23:34:13

Message: 12 of 14

Thanks Tom. Your points are really good. You are absolutely right that the situations in which you might be able to estimate C with any degree of accuracy given the data and a non-linear model will vary greatly.

Perhaps, though I can address your last statement.

" For example, suppose if you decided C had some sort of Weibull
distribution. I don't understand where you would go with that, and how you
would use that information to estimate A and B. But I would consider neither
the mode nor the mean, but perhaps the median (50% point) if I had to
describe C using one numeric value."

The beauty of this discussion is that you are able to retrieve a degree of freedom in your model by estimating C outside of your model and plugging it in.

In the model, y = f(t) = A*exp(-B*t) + C, the parameters are not orthogonal, and so the solution for C will affect the solutions for B and A, if solved for in the model. Moreover, since the problem is ill-posed, a small change in the estimate for C can cause a large change in the estimate for A&B. And since, in the particular setting I posed, sampling is not acquired anywhere near t==infiniti and the decay constant is never sufficiently large to cause the first term to go anywhere near the computational equivalent of zero, there will be significant uncertainty in the estimation of C. As this is a model, where uncertainty in C is propagated to the estimates of B & A, it is less acceptable.

Rather, if the estimate of C, can be attained using data that would otherwise be thrown away, using prior knowledge of its distribution and a maximum likelihood estimate, then one can improve the accuracy of estimate of A & B by not fitting C in the model & also regaining a degree of freedom.

Could you explain why you would choose the median, rather than the mean or mode? The mode would be the most probable value, while the mean would be the expected value. The median might be more robust, but I'm not sure it would work well to plug in for C.

Subject: Expectation Value or Most Probable Value???

From: Tom Lane

Date: 29 Feb, 2012 21:02:57

Message: 13 of 14

> Could you explain why you would choose the median, rather than the mean or
> mode? The mode would be the most probable value, while the mean would be
> the expected value. The median might be more robust, but I'm not sure it
> would work well to plug in for C.

The median is right smack dab in the middle. The mode might be at an
extreme; for example the exponential distribution has a mode of zero. The
mean might be infinite or way out in the tail of a heavy-tailed
distribution.

I'm not claiming the mode or mean would be wrong for you, just why in the
absence of other information I think the median is a safe choice.

-- Tom

Subject: Expectation Value or Most Probable Value???

From: Eric Diaz

Date: 29 Feb, 2012 21:55:23

Message: 14 of 14

Thanks Tom. Yeah, I had a feeling that you were going to say something like that. Your reasoning is perfectly sound, and I have thought similarly myself but the answer still seems elusive.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us