Thread Subject: interpolate missing data

My matrix

a = magic(4); a(2,2) = NaN ; a(3,3)=NaN

a =

    16 2 3 13
     5 NaN 10 8
     9 7 NaN 12
     4 14 15 1

is missing some data, how do I interpolate missing data
along the columns?

"Hydroman S" <amirgsalem@gmail.com> wrote in message
<fvq09u$59o$1@fred.mathworks.com>...
> My matrix
>
> a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
>
> a =
>
> 16 2 3 13
> 5 NaN 10 8
> 9 7 NaN 12
> 4 14 15 1
>
> is missing some data, how do I interpolate missing data
> along the columns?
 
inpaint nans will interpolate in two
dimensions.

http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?
objectId=4551&objectType=file

If you only want to interpolate in one
dimension, then you can most simply loop
over each column, calling inpaint_nans
repeatedly.

Or you could use interp1, but you will
need to do more work, finding the nans
yourself.

John

Thanks John,

Here is what I did, and it seems to have worked, I just
wanted to get your blessing. Also, I was not sure how to
implement a ‘spline’ interpolation using your
inpaint_nans.m function.




a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
[M,N] = size(a);
    for k = 1:length(M)
a=inpaint_nans(a)
    end

a =

    16 2 3 13
     5 NaN 10 8
     9 7 NaN 12
     4 14 15 1


a =

   16.0000 2.0000 3.0000 13.0000
    5.0000 6.0000 10.0000 8.0000
    9.0000 7.0000 11.0000 12.0000
    4.0000 14.0000 15.0000 1.0000

"Hydroman S" <amirgsalem@gmail.com> wrote in message
<fvq8k7$4h2$1@fred.mathworks.com>...
> Thanks John,
>
> Here is what I did, and it seems to have worked, I just
> wanted to get your blessing. Also, I was not sure how to
> implement a ‘spline’ interpolation using your
> inpaint_nans.m function.
>
>
>
>
> a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
> [M,N] = size(a);
> for k = 1:length(M)
> a=inpaint_nans(a)
> end
>
> a =
>
> 16 2 3 13
> 5 NaN 10 8
> 9 7 NaN 12
> 4 14 15 1
>
>
> a =
>
> 16.0000 2.0000 3.0000 13.0000
> 5.0000 6.0000 10.0000 8.0000
> 9.0000 7.0000 11.0000 12.0000
> 4.0000 14.0000 15.0000 1.0000

What you did was equivalent to a single call
to inpaint_nans, working in 2 dimensions.
Note the difference below between b and c.

a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
b=inpaint_nans(a)

[M,N] = size(a);
c = a;
for i = 1:M
  c(:,i) = inpaint_nans(a(:,i));
end
c


b =
           16 2 3 13
            5 6 10 8
            9 7 11 12
            4 14 15 1

c =
           16 2 3 13
            5 3.6 10 8
            9 7 13.4 12
            4 14 15 1

Of course, you need to decide whether
one is more appropriate than the other.
It depends on how the columns of a are
related to each other.

John

Q: Why do I get this error, even with the example
provided?


Warning: Matrix is singular to working precision.
> In inpaint_nans at 170
Warning: Matrix is singular to working precision.
> In inpaint_nans at 170


I don't see any singularities with my matrix. Both det(b)
& det(c) are +ve real numbers

"Hydroman S" <amirgsalem@gmail.com> wrote in message
<fvqclj$81r$1@fred.mathworks.com>...
> Q: Why do I get this error, even with the example
> provided?
>
>
> Warning: Matrix is singular to working precision.
> > In inpaint_nans at 170
> Warning: Matrix is singular to working precision.
> > In inpaint_nans at 170
>
>
> I don't see any singularities with my matrix. Both det(b)
> & det(c) are +ve real numbers

What example generates this error message?
Please show me or send it to me.

Even if all of the elements are NaNs, that
message does not result in my tests.

John

% the example

a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
b=inpaint_nans(a)
[M,N] = size(a);
c = a;
for i = 1:M
  c(:,i) = inpaint_nans(a(:,i));
end
c

if I paste above example in matlab 7.0.0.19920 (R14), this
is the output I get:

a =

    16 2 3 13
     5 NaN 10 8
     9 7 NaN 12
     4 14 15 1


b =

           16 2 3 13
            5 6 10 8
            9 7 11 12
            4 14 15 1

Warning: Matrix is singular to working precision.
> In inpaint_nans at 170
Warning: Matrix is singular to working precision.
> In inpaint_nans at 170

c =

           16 2 3 13
            5 3.6 10 8
            9 7 13.4 12
            4 14 15 1


and if I paste it in matlab 6.1.0.450 (R12.1), this is
what I get:

a =

    16 2 3 13
     5 NaN 10 8
     9 7 NaN 12
     4 14 15 1

??? Error: File: C:\MATLAB6p1\work\inpaint_nans.m Line:
123 Column: 16
Expected a variable, function, or constant, found "|".

"Hydroman S" <amirgsalem@gmail.com> wrote in message
<fvqjf3$mp2$1@fred.mathworks.com>...
> % the example
>
> a = magic(4); a(2,2) = NaN ; a(3,3)=NaN
> b=inpaint_nans(a)
> [M,N] = size(a);
> c = a;
> for i = 1:M
> c(:,i) = inpaint_nans(a(:,i));
> end
> c
>
> if I paste above example in matlab 7.0.0.19920 (R14), this
> is the output I get:
>
> a =
>
> 16 2 3 13
> 5 NaN 10 8
> 9 7 NaN 12
> 4 14 15 1
>
>
> b =
>
> 16 2 3 13
> 5 6 10 8
> 9 7 11 12
> 4 14 15 1
>
> Warning: Matrix is singular to working precision.
> > In inpaint_nans at 170
> Warning: Matrix is singular to working precision.
> > In inpaint_nans at 170
>
> c =
>
> 16 2 3 13
> 5 3.6 10 8
> 9 7 13.4 12
> 4 14 15 1
>
>
> and if I paste it in matlab 6.1.0.450 (R12.1), this is
> what I get:
>
> a =
>
> 16 2 3 13
> 5 NaN 10 8
> 9 7 NaN 12
> 4 14 15 1
>
> ??? Error: File: C:\MATLAB6p1\work\inpaint_nans.m Line:
> 123 Column: 16
> Expected a variable, function, or constant, found "|".

The latter error happened in an older
release of Matlab. Inpaint_nans uses a
facility that was introduced after version
6.1. So it is not backwards compatible to
the older release.

The first message is a warning message,
not an error. I'm surprised that it happens
on that matrix, since I don't get that when
I run this test. You are using an older
release of Matlab than I have, so I must
presume that it is version specific, and
resolved with my newer release. The
result is at least correct despite the
warning message.

John

On May 7, 9:44=A0am, "Hydroman S" <amirgsa...@gmail.com> wrote:
> % the example
>
> a =3D magic(4); a(2,2) =3D NaN ; a(3,3)=3DNaN
> b=3Dinpaint_nans(a)
> [M,N] =3D size(a);
> c =3D a;
> for i =3D 1:M
> =A0 c(:,i) =3D inpaint_nans(a(:,i));
> end
> c
>
> if I paste above example in matlab 7.0.0.19920 (R14), this
> is the output I get:
>
> a =3D
>
> =A0 =A0 16 =A0 =A0 2 =A0 =A0 3 =A0 =A013
> =A0 =A0 =A05 =A0 NaN =A0 =A010 =A0 =A0 8
> =A0 =A0 =A09 =A0 =A0 7 =A0 NaN =A0 =A012
> =A0 =A0 =A04 =A0 =A014 =A0 =A015 =A0 =A0 1
>
> b =3D
>
> =A0 =A0 =A0 =A0 =A0 =A016 =A0 =A0 =A0 =A0 =A0 =A02 =A0 =A0 =A0 =A0 =A0 =A0=
3 =A0 =A0 =A0 =A0 =A0 13
> =A0 =A0 =A0 =A0 =A0 =A0 5 =A0 =A0 =A0 =A0 =A0 =A06 =A0 =A0 =A0 =A0 =A0 10 =
=A0 =A0 =A0 =A0 =A0 =A08
> =A0 =A0 =A0 =A0 =A0 =A0 9 =A0 =A0 =A0 =A0 =A0 =A07 =A0 =A0 =A0 =A0 =A0 11 =
=A0 =A0 =A0 =A0 =A0 12
> =A0 =A0 =A0 =A0 =A0 =A0 4 =A0 =A0 =A0 =A0 =A0 14 =A0 =A0 =A0 =A0 =A0 15 =
=A0 =A0 =A0 =A0 =A0 =A01
>
> Warning: Matrix is singular to working precision.> In inpaint_nans at 170
>
> Warning: Matrix is singular to working precision.
>
> > In inpaint_nans at 170
>
> c =3D
>
> =A0 =A0 =A0 =A0 =A0 =A016 =A0 =A0 =A0 =A0 =A0 =A02 =A0 =A0 =A0 =A0 =A0 =A0=
3 =A0 =A0 =A0 =A0 =A0 13
> =A0 =A0 =A0 =A0 =A0 =A0 5 =A0 =A0 =A0 =A0 =A03.6 =A0 =A0 =A0 =A0 =A0 10 =
=A0 =A0 =A0 =A0 =A0 =A08
> =A0 =A0 =A0 =A0 =A0 =A0 9 =A0 =A0 =A0 =A0 =A0 =A07 =A0 =A0 =A0 =A0 13.4 =
=A0 =A0 =A0 =A0 =A0 12
> =A0 =A0 =A0 =A0 =A0 =A0 4 =A0 =A0 =A0 =A0 =A0 14 =A0 =A0 =A0 =A0 =A0 15 =
=A0 =A0 =A0 =A0 =A0 =A01
>
> and if I paste it in matlab 6.1.0.450 (R12.1), this is
> what I get:
>
> a =3D
>
> =A0 =A0 16 =A0 =A0 2 =A0 =A0 3 =A0 =A013
> =A0 =A0 =A05 =A0 NaN =A0 =A010 =A0 =A0 8
> =A0 =A0 =A09 =A0 =A0 7 =A0 NaN =A0 =A012
> =A0 =A0 =A04 =A0 =A014 =A0 =A015 =A0 =A0 1
>
> ??? Error: File: C:\MATLAB6p1\work\inpaint_nans.m Line:
> 123 Column: 16
> Expected a variable, function, or constant, found "|".

Here is an alternative routine that is much simpler than
inpaint_nans. It works on vectors and simply does linear
interpolation across gaps. It comes from Rich Pawlowicz's t_tide
package:
function y=3Dfixgaps(x);
% FIXGAPS Linearly interpolates gaps in a time series
% YOUT=3DFIXGAPS(YIN) linearly interpolates over NaN
% in the input time series (may be complex), but ignores
% trailing and leading NaN.
%

% R. Pawlowicz 6/Nov/99

y=3Dx;

bd=3Disnan(x);
gd=3Dfind(~bd);

bd([1:(min(gd)-1) (max(gd)+1):end])=3D0;


y(bd)=3Dinterp1(gd,x(gd),find(bd));