MATLAB Central Newsreader - Kalman filtering with multiplicative noise

Kalman filtering with multiplicative noise

dsp@myallit.com — Mon, 21 Jul 2008 02:51:02 -0400

I'm trying to implement a Kalman filter in MATLAB that will use two
types of measurements: volume and in/out flow rate. For the flow rate,
the measurement error is additive Gaussian, but for the volume the
measurement error is expressed as a percentage of the volume, so that
the volume measurement is less accurate when its value is higher. I
think the measurement model should therefore be:

Flow rate measurement model:
z1 = x1 + v1 where v1 ~ N(0,e1)

Volume measurement model:
z2 = x2*v2 where v2 ~ N(1,e2)

I assumed the volume filtering should be done in the log domain to
make the noise additive but how do I deal with a noise mean of one
when the Kalman filter assumes a mean of zero? And how can I have a
Kalman filter using both the measurements if one is in the log domain
and the other one isn't?

I am also dealing with a system where measurements will usually be
missing (they are arriving sequentially) and at an uneven sampling
rate, any other pointers on these too would be appreciated.

Re: Kalman filtering with multiplicative noise

Bruno Luong — Mon, 21 Jul 2008 07:35:02 -0400

dsp@myallit.com wrote in message
<1309743c-3d5c-4165-9cdb-1ab51892727d@z16g2000prn.googlegroups.com>...
> I'm trying to implement a Kalman filter in MATLAB that
will use two
> types of measurements: volume and in/out flow rate. For
the flow rate,
> the measurement error is additive Gaussian, but for the
volume the
> measurement error is expressed as a percentage of the
volume, so that
> the volume measurement is less accurate when its value is
higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 = x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 = x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log
domain to
> make the noise additive but how do I deal with a noise
mean of one
> when the Kalman filter assumes a mean of zero? And how can
I have a
> Kalman filter using both the measurements if one is in the
log domain
> and the other one isn't?

You might consider Extended Kalman filtering (EKF). Be aware
about the eventual non-stability of the scheme.

>
> I am also dealing with a system where measurements will
usually be
> missing (they are arriving sequentially) and at an uneven
sampling
> rate, any other pointers on these too would be appreciated.

No difference in implementation, just change your state
transition matrix accordingly.

Bruno

Re: Kalman filtering with multiplicative noise

RRogers — Mon, 21 Jul 2008 14:22:38 -0400

On Jul 20, 10:51 pm, d...@myallit.com wrote:
> I'm trying to implement a Kalman filter in MATLAB that will use two
> types of measurements: volume and in/out flow rate. For the flow rate,
> the measurement error is additive Gaussian, but for the volume the
> measurement error is expressed as a percentage of the volume, so that
> the volume measurement is less accurate when its value is higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 = x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 = x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log domain to
> make the noise additive but how do I deal with a noise mean of one
> when the Kalman filter assumes a mean of zero? And how can I have a
> Kalman filter using both the measurements if one is in the log domain
> and the other one isn't?
>
> I am also dealing with a system where measurements will usually be
> missing (they are arriving sequentially) and at an uneven sampling
> rate, any other pointers on these too would be appreciated.

A couple of comments:
1) Unless v1,v2 have covariance then they are independent and you can
deal with them separately. It doesn't seem to make any sense to
combine the estimation of z1,z2 in one Kalman filter.
2) In the case of equation two, typically v2 would be restricted to be
>0. I would expect v2=lognormal(1,mu) so that ln(v2)=N(0,mu); or see
comment 4.
3) Given the above then defining nz2=log(z2), nx2=log(x2), nv2=log(v2)
would yield a perfectly normal distribution; sorry for the pun:)
4) Even is v2 wasn't truly log-normal you could try to use the first
and second moments as a description of the real ln(v2) distribution
and see how adequate the description works.

IMHO: all of engineering mathematics is descriptive and the only
question is how adequate the descriptions are.

Ray

Re: Kalman filtering with multiplicative noise

RRogers — Mon, 21 Jul 2008 14:41:29 -0400

On Jul 20, 10:51 pm, d...@myallit.com wrote:
> I'm trying to implement a Kalman filter in MATLAB that will use two
> types of measurements: volume and in/out flow rate. For the flow rate,
> the measurement error is additive Gaussian, but for the volume the
> measurement error is expressed as a percentage of the volume, so that
> the volume measurement is less accurate when its value is higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 = x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 = x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log domain to
> make the noise additive but how do I deal with a noise mean of one
> when the Kalman filter assumes a mean of zero? And how can I have a
> Kalman filter using both the measurements if one is in the log domain
> and the other one isn't?
>
> I am also dealing with a system where measurements will usually be
> missing (they are arriving sequentially) and at an uneven sampling
> rate, any other pointers on these too would be appreciated.

Concerning the uneven sampling there have been discussions of this on
this newsgroup in the past; which I haven't followed carefully.
In my experience the uneven sampling could be rolled into either
multiplicative or additive noise and dealt with as another noise term;
but I had enough design freedom so that I could make do with these
approximations. That is to say, the signals were always "chopped" to
avoid 1/f noise and actual information was contained in small
attenuation of the signals; thus the effect of the changing sampling
point could be estimated/bounded by the chopped signal independent of
the information.

RayR

Ray

Re: Kalman filtering with multiplicative noise

Tim Wescott — Mon, 21 Jul 2008 16:20:27 -0400

On Sun, 20 Jul 2008 19:51:02 -0700, dsp wrote:

> I'm trying to implement a Kalman filter in MATLAB that will use two
> types of measurements: volume and in/out flow rate. For the flow rate,
> the measurement error is additive Gaussian, but for the volume the
> measurement error is expressed as a percentage of the volume, so that
> the volume measurement is less accurate when its value is higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 = x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 = x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log domain to make
> the noise additive but how do I deal with a noise mean of one when the
> Kalman filter assumes a mean of zero? And how can I have a Kalman filter
> using both the measurements if one is in the log domain and the other
> one isn't?
>
> I am also dealing with a system where measurements will usually be
> missing (they are arriving sequentially) and at an uneven sampling rate,
> any other pointers on these too would be appreciated.

As Ray pointed out these seem to be completely independent measurements,
unless V1 and V2 are correlated.

With any affine system, such as the log-domain volume system, you can
subtract out the offset to make the system linear, do your linear
processing, then add the offset back in. I.O.W., calculate log z2 = log
x2 + log v2 + 1, then define dlz2 = log z2 - 1, then find your estimated
dlz2, then back up to your estimated z2. (Note that your estimate of z2
probably won't be strictly optimal in the mean-squared sense, only your
estimate of dlz2 will be. But you'll certainly be closer than you would
by a lot of other means).

--
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html

Re: Kalman filtering with multiplicative noise

dsp@myallit.com — Mon, 21 Jul 2008 23:49:45 -0400

On Jul 22, 2:20=A0am, Tim Wescott <t...@seemywebsite.com> wrote:
> On Sun, 20 Jul 2008 19:51:02 -0700, dsp wrote:
> > I'm trying to implement a Kalman filter in MATLAB that will use two
> > types of measurements: volume and in/out flow rate. For the flow rate,
> > the measurement error is additive Gaussian, but for the volume the
> > measurement error is expressed as a percentage of the volume, so that
> > the volume measurement is less accurate when its value is higher. I
> > think the measurement model should therefore be:
>
> > Flow rate measurement model:
> > z1 =3D x1 + v1 where v1 ~ N(0,e1)
>
> > Volume measurement model:
> > z2 =3D x2*v2 where v2 ~ N(1,e2)
>
> > I assumed the volume filtering should be done in the log domain to make
> > the noise additive but how do I deal with a noise mean of one when the
> > Kalman filter assumes a mean of zero? And how can I have a Kalman filte=
r
> > using both the measurements if one is in the log domain and the other
> > one isn't?
>
> > I am also dealing with a system where measurements will usually be
> > missing (they are arriving sequentially) and at an uneven sampling rate=
,
> > any other pointers on these too would be appreciated.
>
> As Ray pointed out these seem to be completely independent measurements,
> unless V1 and V2 are correlated.
>
> With any affine system, such as the log-domain volume system, you can
> subtract out the offset to make the system linear, do your linear
> processing, then add the offset back in. =A0I.O.W., calculate log z2 =3D =
log
> x2 + log v2 + 1, then define dlz2 =3D log z2 - 1, then find your estimate=
d
> dlz2, then back up to your estimated z2. =A0(Note that your estimate of z=
2
> probably won't be strictly optimal in the mean-squared sense, only your
> estimate of dlz2 will be. =A0But you'll certainly be closer than you woul=
d
> by a lot of other means).
>
> --
> Tim Wescott
> Control systems and communications consultinghttp://www.wescottdesign.com
>
> Need to learn how to apply control theory in your embedded system?
> "Applied Control Theory for Embedded Systems" by Tim Wescott
> Elsevier/Newnes,http://www.wescottdesign.com/actfes/actfes.html

My state transition for these two measurements is given by:

F =3D [dt 1
1 0]

where dt is the time step, so as far as I understand it these
measurements are correlated - if z1 is positive, then z2 will be
increasing and vice versa.

Doing this in the log domain makes sense, but how can I have one
measurement in the log domain and the other not? The state transition
won't make sense.

Equation 2 will actually reach zero quite often, so maybe I need
multiplicative and additive noise in my measurement model for z2, not
too sure about that.

Re: Kalman filtering with multiplicative noise

dsp@myallit.com — Tue, 22 Jul 2008 00:06:45 -0400

On Jul 21, 5:35 pm, "Bruno Luong" <b.lu...@fogale.fr> wrote:
> d...@myallit.com wrote in message
>
> You might consider Extended Kalman filtering (EKF). Be aware
> about the eventual non-stability of the scheme.
>

What do you mean by the eventual non-stability? I did look at the EKF,
there is some simple sample MATLAB code here:
http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=18189

But in the first few lines of this script it says:

% for nonlinear dynamic system:
% x_k+1 = f(x_k) + w_k
% z_k = h(x_k) + v_k
% where w ~ N(0,Q) meaning w is gaussian noise with covariance Q
% v ~ N(0,R) meaning v is gaussian noise with covariance R

so the EKF looks appropriate for non-linear process models and
measurement models that can be represented by any arbitrary functions
f(x) and h(x), but the noise is still assumed to be additive.

Re: Kalman filtering with multiplicative noise

Tim Wescott — Tue, 22 Jul 2008 01:06:03 -0400

dsp@myallit.com wrote:
> On Jul 22, 2:20 am, Tim Wescott <t...@seemywebsite.com> wrote:
>> On Sun, 20 Jul 2008 19:51:02 -0700, dsp wrote:
>>> I'm trying to implement a Kalman filter in MATLAB that will use two
>>> types of measurements: volume and in/out flow rate. For the flow rate,
>>> the measurement error is additive Gaussian, but for the volume the
>>> measurement error is expressed as a percentage of the volume, so that
>>> the volume measurement is less accurate when its value is higher. I
>>> think the measurement model should therefore be:
>>> Flow rate measurement model:
>>> z1 = x1 + v1 where v1 ~ N(0,e1)
>>> Volume measurement model:
>>> z2 = x2*v2 where v2 ~ N(1,e2)
>>> I assumed the volume filtering should be done in the log domain to make
>>> the noise additive but how do I deal with a noise mean of one when the
>>> Kalman filter assumes a mean of zero? And how can I have a Kalman filter
>>> using both the measurements if one is in the log domain and the other
>>> one isn't?
>>> I am also dealing with a system where measurements will usually be
>>> missing (they are arriving sequentially) and at an uneven sampling rate,
>>> any other pointers on these too would be appreciated.
>> As Ray pointed out these seem to be completely independent measurements,
>> unless V1 and V2 are correlated.
>>
>> With any affine system, such as the log-domain volume system, you can
>> subtract out the offset to make the system linear, do your linear
>> processing, then add the offset back in. I.O.W., calculate log z2 = log
>> x2 + log v2 + 1, then define dlz2 = log z2 - 1, then find your estimated
>> dlz2, then back up to your estimated z2. (Note that your estimate of z2
>> probably won't be strictly optimal in the mean-squared sense, only your
>> estimate of dlz2 will be. But you'll certainly be closer than you would
>> by a lot of other means).
>>
>> --
>> Tim Wescott
>> Control systems and communications consultinghttp://www.wescottdesign.com
>>
>> Need to learn how to apply control theory in your embedded system?
>> "Applied Control Theory for Embedded Systems" by Tim Wescott
>> Elsevier/Newnes,http://www.wescottdesign.com/actfes/actfes.html
>
> My state transition for these two measurements is given by:
>
> F = [dt 1
> 1 0]
>
> where dt is the time step, so as far as I understand it these
> measurements are correlated - if z1 is positive, then z2 will be
> increasing and vice versa.
>
> Doing this in the log domain makes sense, but how can I have one
> measurement in the log domain and the other not? The state transition
> won't make sense.
>
> Equation 2 will actually reach zero quite often, so maybe I need
> multiplicative and additive noise in my measurement model for z2, not
> too sure about that.

You probably need an extended Kalman filter, where you use your current
best guess of the flow to update the covariance. This will be more
computationally intensive because you won't be able to pre-calculate
your covariance, and it'll be nonlinear because you'll be multiplying by
one of your states.

That's about as much as I know about the subject, so try google, or get
a copy of "Optimal State Estimation" by Simon and have fun.

Alternately, can you do a good enough job with good ol' low-pass filters?

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

Re: Kalman filtering with multiplicative noise

Tim Wescott — Tue, 22 Jul 2008 01:11:28 -0400

dsp@myallit.com wrote:
> On Jul 21, 5:35 pm, "Bruno Luong" <b.lu...@fogale.fr> wrote:
>> d...@myallit.com wrote in message
>>
>> You might consider Extended Kalman filtering (EKF). Be aware
>> about the eventual non-stability of the scheme.
>>
>
> What do you mean by the eventual non-stability? I did look at the EKF,
> there is some simple sample MATLAB code here:
> http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=18189
>
> But in the first few lines of this script it says:
>
> % for nonlinear dynamic system:
> % x_k+1 = f(x_k) + w_k
> % z_k = h(x_k) + v_k
> % where w ~ N(0,Q) meaning w is gaussian noise with covariance Q
> % v ~ N(0,R) meaning v is gaussian noise with covariance R
>
> so the EKF looks appropriate for non-linear process models and
> measurement models that can be represented by any arbitrary functions
> f(x) and h(x), but the noise is still assumed to be additive.

He means that it's a nonlinear system, and therefore you can't assume
that just because it's locally stable around some state that it's
globally stable for any state. In particular, there may be input
vectors that will drive it into a limit cycle or off toward infinity,
either permanently or temporarily.

Contrary to the beliefs of some, the MathWorks isn't on high, and MatLab
isn't an extension of the Bible. I would trust them for examples, but
don't take their documentation for anything but a means of selling you
copies of MatLab.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

Re: Kalman filtering with multiplicative noise

pnachtwey — Tue, 22 Jul 2008 03:39:24 -0400

On Jul 20, 7:51=A0pm, d...@myallit.com wrote:
> I'm trying to implement a Kalman filter in MATLAB that will use two
> types of measurements: volume and in/out flow rate. For the flow rate,
> the measurement error is additive Gaussian, but for the volume the
> measurement error is expressed as a percentage of the volume, so that
> the volume measurement is less accurate when its value is higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 =3D x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 =3D x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log domain to
> make the noise additive but how do I deal with a noise mean of one
> when the Kalman filter assumes a mean of zero? And how can I have a
> Kalman filter using both the measurements if one is in the log domain
> and the other one isn't?
>
> I am also dealing with a system where measurements will usually be
> missing (they are arriving sequentially) and at an uneven sampling
> rate, any other pointers on these too would be appreciated.

If this is a student project then why not assume there will be a set
point and the controller will maintain that set point with little
variation. Now the volume is fixed and the variance is therefore
fixed. Now you can assume the system is linear around that set point.

Peter Nachtwey

Re: Kalman filtering with multiplicative noise

RRogers — Tue, 22 Jul 2008 12:54:42 -0400

On Jul 21, 8:06 pm, d...@myallit.com wrote:
> On Jul 21, 5:35 pm, "Bruno Luong" <b.lu...@fogale.fr> wrote:
>
> > d...@myallit.com wrote in message
>
> > You might consider Extended Kalman filtering (EKF). Be aware
> > about the eventual non-stability of the scheme.
>
> What do you mean by the eventual non-stability? I did look at the EKF,
> there is some simple sample MATLAB code here:http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objec...
>
> But in the first few lines of this script it says:
>
> % for nonlinear dynamic system:
> % x_k+1 = f(x_k) + w_k
> % z_k = h(x_k) + v_k
> % where w ~ N(0,Q) meaning w is gaussian noise with covariance Q
> % v ~ N(0,R) meaning v is gaussian noise with covariance R
>
> so the EKF looks appropriate for non-linear process models and
> measurement models that can be represented by any arbitrary functions
> f(x) and h(x), but the noise is still assumed to be additive.

If you change the second equation to log's, and if the log(v2) was
adequately described by a normal distribution then the noise would be
additive and the non-linearity would pushed into the x2/z2 and it
seems that the EKF could deal with it. In terms of the wikipedia
article and your state equation
z1=x1+v1
lz2=lx2+lv2

and presumptively
x1'=dt*lx1+exp(lx2)+v1
lx2'=log(x1)+lv2

You should really presume these state equations are wrong! Without
the differential equations including the noise terms I really can't
get a handle on them mentally ( I usually get the first go round wrong
anyway).

RayR

Re: Kalman filtering with multiplicative noise

RRogers — Tue, 22 Jul 2008 13:11:14 -0400

On Jul 21, 7:49 pm, d...@myallit.com wrote:
> On Jul 22, 2:20 am, Tim Wescott <t...@seemywebsite.com> wrote:
>
>
>
> > On Sun, 20 Jul 2008 19:51:02 -0700, dsp wrote:
> > > I'm trying to implement a Kalman filter in MATLAB that will use two
> > > types of measurements: volume and in/out flow rate. For the flow rate,
> > > the measurement error is additive Gaussian, but for the volume the
> > > measurement error is expressed as a percentage of the volume, so that
> > > the volume measurement is less accurate when its value is higher. I
> > > think the measurement model should therefore be:
>
> > > Flow rate measurement model:
> > > z1 = x1 + v1 where v1 ~ N(0,e1)
>
> > > Volume measurement model:
> > > z2 = x2*v2 where v2 ~ N(1,e2)
>
> > > I assumed the volume filtering should be done in the log domain to make
> > > the noise additive but how do I deal with a noise mean of one when the
> > > Kalman filter assumes a mean of zero? And how can I have a Kalman filter
> > > using both the measurements if one is in the log domain and the other
> > > one isn't?
>
> > > I am also dealing with a system where measurements will usually be
> > > missing (they are arriving sequentially) and at an uneven sampling rate,
> > > any other pointers on these too would be appreciated.
>
> > As Ray pointed out these seem to be completely independent measurements,
> > unless V1 and V2 are correlated.
>
> > With any affine system, such as the log-domain volume system, you can
> > subtract out the offset to make the system linear, do your linear
> > processing, then add the offset back in. I.O.W., calculate log z2 = log
> > x2 + log v2 + 1, then define dlz2 = log z2 - 1, then find your estimated
> > dlz2, then back up to your estimated z2. (Note that your estimate of z2
> > probably won't be strictly optimal in the mean-squared sense, only your
> > estimate of dlz2 will be. But you'll certainly be closer than you would
> > by a lot of other means).
>
> > --
> > Tim Wescott
> > Control systems and communications consultinghttp://www.wescottdesign.com
>
> > Need to learn how to apply control theory in your embedded system?
> > "Applied Control Theory for Embedded Systems" by Tim Wescott
> > Elsevier/Newnes,http://www.wescottdesign.com/actfes/actfes.html
>
> My state transition for these two measurements is given by:
>
> F = [dt 1
> 1 0]
>
> where dt is the time step, so as far as I understand it these
> measurements are correlated - if z1 is positive, then z2 will be
> increasing and vice versa.
>
> Doing this in the log domain makes sense, but how can I have one
> measurement in the log domain and the other not? The state transition
> won't make sense.
>
> Equation 2 will actually reach zero quite often, so maybe I need
> multiplicative and additive noise in my measurement model for z2, not
> too sure about that.

I really suggest that you make up a graphical model of the system,
Simulink or Scicos ( I can deal with Scicos), both the physical system
and then the proposed filtering.

RayR

Re: Kalman filtering with multiplicative noise

illywhacker — Tue, 22 Jul 2008 13:44:55 -0400

On Jul 21, 4:51=A0am, d...@myallit.com wrote:
> I'm trying to implement a Kalman filter in MATLAB that will use two
> types of measurements: volume and in/out flow rate. For the flow rate,
> the measurement error is additive Gaussian, but for the volume the
> measurement error is expressed as a percentage of the volume, so that
> the volume measurement is less accurate when its value is higher. I
> think the measurement model should therefore be:
>
> Flow rate measurement model:
> z1 =3D x1 + v1 where v1 ~ N(0,e1)
>
> Volume measurement model:
> z2 =3D x2*v2 where v2 ~ N(1,e2)
>
> I assumed the volume filtering should be done in the log domain to
> make the noise additive but how do I deal with a noise mean of one
> when the Kalman filter assumes a mean of zero? And how can I have a
> Kalman filter using both the measurements if one is in the log domain
> and the other one isn't?
>
> I am also dealing with a system where measurements will usually be
> missing (they are arriving sequentially) and at an uneven sampling
> rate, any other pointers on these too would be appreciated.

So your multiplicative noise can be negative? This seems strange.

In any case, why not simply derive the appropriate filtering from
scratch instead of trying to find off the shelf solutions? That way
you can incorporate all the specifics of your problem.

illywhacker;

Re: Kalman filtering with multiplicative noise

stanp — Tue, 22 Jul 2008 17:18:06 -0400

On Jul 21, 8:06 pm, d...@myallit.com wrote:
> On Jul 21, 5:35 pm, "Bruno Luong" <b.lu...@fogale.fr> wrote:
>
> > d...@myallit.com wrote in message
>
> > You might consider Extended Kalman filtering (EKF). Be aware
> > about the eventual non-stability of the scheme.
>
> What do you mean by the eventual non-stability? I did look at the EKF,
> there is some simple sample MATLAB code here:http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objec...
>
> But in the first few lines of this script it says:
>
> % for nonlinear dynamic system:
> % x_k+1 = f(x_k) + w_k
> % z_k = h(x_k) + v_k
> % where w ~ N(0,Q) meaning w is gaussian noise with covariance Q
> % v ~ N(0,R) meaning v is gaussian noise with covariance R
>
> so the EKF looks appropriate for non-linear process models and
> measurement models that can be represented by any arbitrary functions
> f(x) and h(x), but the noise is still assumed to be additive.

The EKF propagates first and second moments which is essentially
treating the problem like it can be approximated like a linear
Gaussian.

Particle filters come in various flavors and there are versions that
can handle multiplicative noise.

Re: Kalman filtering with multiplicative noise

dsp@myallit.com — Wed, 23 Jul 2008 05:02:14 -0400

> Contrary to the beliefs of some, the MathWorks isn't on high, and MatLab
> isn't an extension of the Bible. I would trust them for examples, but
> don't take their documentation for anything but a means of selling you
> copies of MatLab.

The link I posted is a user contribution, not an example from the
Mathworks, so I don't trust it entirely. However the Wikipedia article
on EKF's seems to show the same formulation with the noise being
assumed to be additive, only the process and measurement models are
allowed to be non-linear:

http://en.wikipedia.org/wiki/Extended_Kalman_filter#Formulation

Does this mean the EKF is not appropriate for my problem or am I
misunderstanding how it works?

> If this is a student project then why not assume there will be a set
> point and the controller will maintain that set point with little
> variation. Now the volume is fixed and the variance is therefore
> fixed. Now you can assume the system is linear around that set point.

I have already implemented a regular Kalman filter simply by
approximating both of the noise terms as additive noise and the
results are okay but not great (another idea I might try is
approximating both of them as multiplicative noise, and then I can do
all the calculations in the log domain). To improve it I want to try
incorporating the multiplicative noise. When z2 terms have smaller
values, we have more confidence in them (less variance), so I want to
explicitly incorporate this into the filter. In particular, when z2
reaches zero, we have almost 100% confidence it is correct.

> I really suggest that you make up a graphical model of the system,
> Simulink or Scicos ( I can deal with Scicos), both the physical system
> and then the proposed filtering.

I'm not too familiar with Simulink or Scicos but here's a small
diagram:

             z2
        +---------+
        | |
======>| |
   z1 | |
<====== | |
        | |
        +---------+

The measurement z1 is measuring the amount going in and out of the
container (flow rate), the measurement z2 is measuring the total
currently in the container (volume). I am trying to find the true
value of x2 (the volume) by combining the noisy measurements of z1 and
z2. Actually there are multiple sensors measuring flow rate and
volume, but I've only shown one of each to simplify the problem. The
z1 measurement has additive Gaussian noise, the z2 measurement has
multiplicative (and possibly additive?) Gaussian noise.

> So your multiplicative noise can be negative? This seems strange.

> In any case, why not simply derive the appropriate filtering from
> scratch instead of trying to find off the shelf solutions? That way
> you can incorporate all the specifics of your problem.

No I suppose it should always should be positive, I have uploaded an
example image to help clarify:

http://picasaweb.google.com.au/dspmyallit/ExampleVolumeMeasurements/photo#5226062785340902530

> The EKF propagates first and second moments which is essentially
> treating the problem like it can be approximated like a linear
> Gaussian.

> Particle filters come in various flavors and there are versions that
> can handle multiplicative noise.

I will look into particle filters. Are you aware of any good C++
implementations? (For the Kalman filter I have been using MATLAB to
design and test it, OpenCV to implement it.)

Re: Kalman filtering with multiplicative noise

Tim Wescott — Wed, 23 Jul 2008 06:16:47 -0400

dsp@myallit.com wrote:
>> Contrary to the beliefs of some, the MathWorks isn't on high, and MatLab
>> isn't an extension of the Bible. I would trust them for examples, but
>> don't take their documentation for anything but a means of selling you
>> copies of MatLab.
>
> The link I posted is a user contribution, not an example from the
> Mathworks, so I don't trust it entirely. However the Wikipedia article
> on EKF's seems to show the same formulation with the noise being
> assumed to be additive, only the process and measurement models are
> allowed to be non-linear:
>
> http://en.wikipedia.org/wiki/Extended_Kalman_filter#Formulation
>
> Does this mean the EKF is not appropriate for my problem or am I
> misunderstanding how it works?

The 'E' in 'EKF' stands for 'extended'. I'd take that and run with it.
If you understand how the Kalman filter works you know how to iterate
your covariance matrix with known noise properties -- why not use the
predicted volume to set the noise intensity from one iteration to the next?

>> If this is a student project then why not assume there will be a set
>> point and the controller will maintain that set point with little
>> variation. Now the volume is fixed and the variance is therefore
>> fixed. Now you can assume the system is linear around that set point.
>
> I have already implemented a regular Kalman filter simply by
> approximating both of the noise terms as additive noise and the
> results are okay but not great (another idea I might try is
> approximating both of them as multiplicative noise, and then I can do
> all the calculations in the log domain). To improve it I want to try
> incorporating the multiplicative noise. When z2 terms have smaller
> values, we have more confidence in them (less variance), so I want to
> explicitly incorporate this into the filter. In particular, when z2
> reaches zero, we have almost 100% confidence it is correct.

Good for you -- you're not just using the fancy math because it's there,
you're using it because you have to.

(responses to other guy snipped)

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

Re: Kalman filtering with multiplicative noise

Bruno Luong — Wed, 23 Jul 2008 06:54:02 -0400

dsp@myallit.com wrote in message
<1d3da6d4-ec1f-4e99-ae7e-c3478412f54c@l8g2000prm.googlegroups.com>...
> On Jul 21, 5:35 pm, "Bruno Luong" <b.lu...@fogale.fr> wrote:
> > d...@myallit.com wrote in message
> >
> > You might consider Extended Kalman filtering (EKF). Be aware
> > about the eventual non-stability of the scheme.
> >
>
> What do you mean by the eventual non-stability? I did look
at the EKF,

Tim has answered this question. Mostly the matrix built by
the scheme can diverge, because no theoretical framework
could warranty the stability of EKF, and that has been
observed in the practice.

> so the EKF looks appropriate for non-linear process models and
> measurement models that can be represented by any
arbitrary functions
> f(x) and h(x), but the noise is still assumed to be additive.

Here is an heuristic way to look at, when you have a process of:

x_k = f(x_k-1,u_k) * w_k

it is like

x_k = f(x_k-1,u_k) + (w_k-1)*f

Consider this as your additive noise: f*(w_k-1)
(proportional to the amplitude).

- A better way if to take a log of the state equation

log(x_k) = log[f(x_k-1,u-k)] + log(w_k)

And that's how EKF can be used to cope with the non linearitity.

Bruno

Re: Kalman filtering with multiplicative noise

RRogers — Wed, 23 Jul 2008 14:35:40 -0400

On Jul 23, 1:02 am, d...@myallit.com wrote:
> > Contrary to the beliefs of some, the MathWorks isn't on high, and MatLab
> > isn't an extension of the Bible. I would trust them for examples, but
> > don't take their documentation for anything but a means of selling you
> > copies of MatLab.
>
> The link I posted is a user contribution, not an example from the
> Mathworks, so I don't trust it entirely. However the Wikipedia article
> on EKF's seems to show the same formulation with the noise being
> assumed to be additive, only the process and measurement models are
> allowed to be non-linear:
>
> http://en.wikipedia.org/wiki/Extended_Kalman_filter#Formulation
>
> Does this mean the EKF is not appropriate for my problem or am I
> misunderstanding how it works?
>
> > If this is a student project then why not assume there will be a set
> > point and the controller will maintain that set point with little
> > variation. Now the volume is fixed and the variance is therefore
> > fixed. Now you can assume the system is linear around that set point.
>
> I have already implemented a regular Kalman filter simply by
> approximating both of the noise terms as additive noise and the
> results are okay but not great (another idea I might try is
> approximating both of them as multiplicative noise, and then I can do
> all the calculations in the log domain). To improve it I want to try
> incorporating the multiplicative noise. When z2 terms have smaller
> values, we have more confidence in them (less variance), so I want to
> explicitly incorporate this into the filter. In particular, when z2
> reaches zero, we have almost 100% confidence it is correct.
>
> > I really suggest that you make up a graphical model of the system,
> > Simulink or Scicos ( I can deal with Scicos), both the physical system
> > and then the proposed filtering.
>
> I'm not too familiar with Simulink or Scicos but here's a small
> diagram:
>
> z2
> +---------+
> | |
> ======>| |
> z1 | |
> <====== | |
> | |
> +---------+
>
> The measurement z1 is measuring the amount going in and out of the
> container (flow rate), the measurement z2 is measuring the total
> currently in the container (volume). I am trying to find the true
> value of x2 (the volume) by combining the noisy measurements of z1 and
> z2. Actually there are multiple sensors measuring flow rate and
> volume, but I've only shown one of each to simplify the problem. The
> z1 measurement has additive Gaussian noise, the z2 measurement has
> multiplicative (and possibly additive?) Gaussian noise.
>
> > So your multiplicative noise can be negative? This seems strange.
> > In any case, why not simply derive the appropriate filtering from
> > scratch instead of trying to find off the shelf solutions? That way
> > you can incorporate all the specifics of your problem.
>
> No I suppose it should always should be positive, I have uploaded an
> example image to help clarify:
>
> http://picasaweb.google.com.au/dspmyallit/ExampleVolumeMeasurements/p...
>
> > The EKF propagates first and second moments which is essentially
> > treating the problem like it can be approximated like a linear
> > Gaussian.
> > Particle filters come in various flavors and there are versions that
> > can handle multiplicative noise.
>
> I will look into particle filters. Are you aware of any good C++
> implementations? (For the Kalman filter I have been using MATLAB to
> design and test it, OpenCV to implement it.)
> So your multiplicative noise can be negative? This seems strange.
> > In any case, why not simply derive the appropriate filtering from
> > scratch instead of trying to find off the shelf solutions? That way
> > you can incorporate all the specifics of your problem.
>
> No I suppose it should always should be positive, I have uploaded an
> example image to help clarify:
>
> http://picasaweb.google.com.au/dspmyallit/ExampleVolumeMeasurements/p...

Example? Where did this data come from? If it's experimental data,
can you post the raw numbers. I can characterize it if you want.

RayR