Thread Subject: Matrix inversion problem - What's going on?

Dear all

I've uploaded this pdf-file:
http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html

As you can see, I'm using Matlab:

1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
and b=[1;2;3], meaning that:
2) x=A\b can easily be found
3) A*x = b, right? NO! Not in the second case!

Any hints/explanations are most welcome. I've put some comments in the
uploaded pdf-file and I guess that those of you who'll answer knows
everything about illposed problems, SVD and possible some other things
that I don't know of. I also tried to find the conditioning number for
the two matrices, however I'm a bit lost and can't come up with a good
explanation myself.

"printer newbie!" <newsboost@gmail.com> wrote in message <82709a89-8b9f-41ca-90b4-c9738d23a8c2@d11g2000yqo.googlegroups.com>...
> Dear all
>
> I've uploaded this pdf-file:
> http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html
>
> As you can see, I'm using Matlab:
>
> 1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
> and b=[1;2;3], meaning that:
> 2) x=A\b can easily be found
> 3) A*x = b, right? NO! Not in the second case!


x=A\b
b=A*x

"printer newbie!" <newsboost@gmail.com> wrote in message <82709a89-8b9f-41ca-90b4-c9738d23a8c2@d11g2000yqo.googlegroups.com>...
> Dear all
>
> I've uploaded this pdf-file:
> http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html
>
> As you can see, I'm using Matlab:
>
> 1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
> and b=[1;2;3], meaning that:
> 2) x=A\b can easily be found
> 3) A*x = b, right? NO! Not in the second case!
>
> Any hints/explanations are most welcome. I've put some comments in the
> uploaded pdf-file and I guess that those of you who'll answer knows
> everything about illposed problems, SVD and possible some other things
> that I don't know of. I also tried to find the conditioning number for
> the two matrices, however I'm a bit lost and can't come up with a good
> explanation myself.
>
- - - - - - - -
  With A a 3 x 2 matrix and b a 3 x 1 vector, there is no reason to expect that x = A\b will satisfy A*x = b exactly. You would be expecting to solve three equations with two unknowns, which is generally impossible. All the backslash operator does is find the least squares difference for A*x-b, which of course will usually be nonzero.

Roger Stafford

"printer newbie!" <newsboost@gmail.com> wrote in message <82709a89-8b9f-41ca-90b4-c9738d23a8c2@d11g2000yqo.googlegroups.com>...
> Dear all
>
> I've uploaded this pdf-file:
> http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html
>
> As you can see, I'm using Matlab:
>
> 1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
> and b=[1;2;3], meaning that:
> 2) x=A\b can easily be found
> 3) A*x = b, right? NO! Not in the second case!
>
> Any hints/explanations are most welcome. I've put some comments in the
> uploaded pdf-file and I guess that those of you who'll answer knows
> everything about illposed problems, SVD and possible some other things
> that I don't know of. I also tried to find the conditioning number for
> the two matrices, however I'm a bit lost and can't come up with a good
> explanation myself.
>
As it was said above, this is an overdetermined system. If you are looking for an unique solution, then you system must have one of its lines equal to a linear combination of the two others.
In general if you have p equations and n unknowns (with p>n for overdetermined), your system must have n linear independent equations, hence p-n linear dependent eq.

Regards,
Florin

On 11 Apr., 18:41, "Florin Neacsu" <fneac...@gmail.com> wrote:
> "printer newbie!" <newsbo...@gmail.com> wrote in message <82709a89-8b9f-41ca-90b4-c9738d23a...@d11g2000yqo.googlegroups.com>...
> > Dear all
>
> > I've uploaded this pdf-file:
> >http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html
>
> > As you can see, I'm using Matlab:
>
> > 1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
> > and b=[1;2;3], meaning that:
> > 2) x=A\b can easily be found
> > 3) A*x = b, right? NO! Not in the second case!
>
> > Any hints/explanations are most welcome. I've put some comments in the
> > uploaded pdf-file and I guess that those of you who'll answer knows
> > everything about illposed problems, SVD and possible some other things
> > that I don't know of. I also tried to find the conditioning number for
> > the two matrices, however I'm a bit lost and can't come up with a good
> > explanation myself.
>
> As it was said above, this is an overdetermined system. If you are looking for an unique solution, then you system must have one of its lines equal to a linear combination of the two others.
> In general if you have p equations and n unknowns (with p>n for overdetermined), your system must have n linear independent equations, hence p-n linear dependent eq.

Ok, thanks. So the number of rows is the number of equations and the
number of columns is the number of unknowns...

So it was just a coincidence that it worked out for the first matrix,
but not the second...

What property (of the matrix A) is it then, that makes it work out
great for the first A-matrix, when they're both the same physical size
(3x2)? Are any lines in the first matrix linear combinations of
another line?

"printer newbie!" <newsboost@gmail.com> wrote in message <1ce4d6d3-488e-4d04-a153-a7137ff652b3@l39g2000yqh.googlegroups.com>...
> On 11 Apr., 18:41, "Florin Neacsu" <fneac...@gmail.com> wrote:
> > "printer newbie!" <newsbo...@gmail.com> wrote in message <82709a89-8b9f-41ca-90b4-c9738d23a...@d11g2000yqo.googlegroups.com>...
> > > Dear all
> >
> > > I've uploaded this pdf-file:
> > >http://www.2shared.com/document/o9q5B5Rw/Matlab_matrix_inversion.html
> >
> > > As you can see, I'm using Matlab:
> >
> > > 1) with two different matrices: A=[1 2; 3 4;5 6] and A=[1 2; 3 4;5 5]
> > > and b=[1;2;3], meaning that:
> > > 2) x=A\b can easily be found
> > > 3) A*x = b, right? NO! Not in the second case!
> >
> > > Any hints/explanations are most welcome. I've put some comments in the
> > > uploaded pdf-file and I guess that those of you who'll answer knows
> > > everything about illposed problems, SVD and possible some other things
> > > that I don't know of. I also tried to find the conditioning number for
> > > the two matrices, however I'm a bit lost and can't come up with a good
> > > explanation myself.
> >
> > As it was said above, this is an overdetermined system. If you are looking for an unique solution, then you system must have one of its lines equal to a linear combination of the two others.
> > In general if you have p equations and n unknowns (with p>n for overdetermined), your system must have n linear independent equations, hence p-n linear dependent eq.
>
> Ok, thanks. So the number of rows is the number of equations and the
> number of columns is the number of unknowns...
>
> So it was just a coincidence that it worked out for the first matrix,
> but not the second...
>
> What property (of the matrix A) is it then, that makes it work out
> great for the first A-matrix, when they're both the same physical size
> (3x2)? Are any lines in the first matrix linear combinations of
> another line?
>

In the second case, b (the right hand side) was an
exact linear combination of the columns of your first
matrix A. So you might notice that A1(:,2)/2 would
yield [1;2;3]. This just happens to be identically b.

A way to identify that this happens is when

   rank(A) == rank([A,b])

Try it.

A1=[1 2; 3 4;5 6];
A2=[1 2; 3 4;5 5];
b=[1;2;3];

rank(A1) == rank([A1,b])
ans =
     1

rank(A2) == rank([A2,b])
ans =
     0

Therefore the first problem x = A1\b will have an exact
solution, while the second problem does not. Well,
exact to within the floating point precision of MATLAB.

HTH,
John

On 12 Apr., 13:13, "John D'Errico" <woodch...@rochester.rr.com> wrote:
> "printer newbie!" <newsbo...@gmail.com> wrote in message <1ce4d6d3-488e-4d04-a153-a7137ff65...@l39g2000yqh.googlegroups.com>...

> > So it was just a coincidence that it worked out for the first matrix,
> > but not the second...
>
> > What property (of the matrix A) is it then, that makes it work out
> > great for the first A-matrix, when they're both the same physical size
> > (3x2)? Are any lines in the first matrix linear combinations of
> > another line?
>
> In the second case, b (the right hand side) was an
> exact linear combination of the columns of your first
> matrix A. So you might notice that A1(:,2)/2 would
> yield [1;2;3]. This just happens to be identically b.
>
> A way to identify that this happens is when
>
>    rank(A) == rank([A,b])
>
> Try it.
>
> A1=[1 2; 3 4;5 6];
> A2=[1 2; 3 4;5 5];
> b=[1;2;3];
>
> rank(A1) == rank([A1,b])
> ans =
>      1
>
> rank(A2) == rank([A2,b])
> ans =
>      0
>
> Therefore the first problem x = A1\b will have an exact
> solution, while the second problem does not. Well,
> exact to within the floating point precision of MATLAB.

Great, thanks!

So it had nothing to do with SVD - btw: I'm not that good at singular
value decomposition.

Maybe a clever and friendly guy can help explaining if I can use SVD
to determine the quality of my matrices and whether they're
illconditioned or not...?

On Apr 12, 1:47 pm, "printer newbie!" <newsbo...@gmail.com> wrote:
> On 12 Apr., 13:13, "John D'Errico" <woodch...@rochester.rr.com> wrote:
> > "printer newbie!" <newsbo...@gmail.com> wrote in message <1ce4d6d3-488e-4d04-a153-a7137ff65...@l39g2000yqh.googlegroups.com>...
> > > So it was just a coincidence that it worked out for the first matrix,
> > > but not the second...
>
> > > What property (of the matrix A) is it then, that makes it work out
> > > great for the first A-matrix, when they're both the same physical size
> > > (3x2)? Are any lines in the first matrix linear combinations of
> > > another line?
>
> > In the second case, b (the right hand side) was an
> > exact linear combination of the columns of your first
> > matrix A. So you might notice that A1(:,2)/2 would
> > yield [1;2;3]. This just happens to be identically b.
>
> > A way to identify that this happens is when
>
> >    rank(A) == rank([A,b])
>
> > Try it.
>
> > A1=[1 2; 3 4;5 6];
> > A2=[1 2; 3 4;5 5];
> > b=[1;2;3];
>
> > rank(A1) == rank([A1,b])
> > ans =
> >      1
>
> > rank(A2) == rank([A2,b])
> > ans =
> >      0
>
> > Therefore the first problem x = A1\b will have an exact
> > solution, while the second problem does not. Well,
> > exact to within the floating point precision of MATLAB.
>
> Great, thanks!
>
> So it had nothing to do with SVD

Incorrect. It is intimately related.

1. rank(A,tol) is the number of singular values greater than tol
2. The use of rank(A) uses the default

    tol = max(size(A)) * max(svd(A)) * eps

3. cond(A) is the ratio max(svd(A) / min(svd(A))


>- btw: I'm not that good at singular
> value decomposition.
>
> Maybe a clever and friendly guy can help explaining if I can use SVD
> to determine the quality of my matrices and whether they're
> illconditioned or not

You can either use the above relations or skip the middleman
and just use rank and cond.

See

help rank
doc rank

help norm
doc norm

help cond
doc cond

help svd
doc svd

as well as the corresponding wikipedia explanations.

Reply if you have additional questions.

Hope this helps.

Greg