Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
dot product and vectors with imaginary numbers

Subject: dot product and vectors with imaginary numbers

From: Linda Davis

Date: 6 Jul, 2010 23:59:04

Message: 1 of 18

Ok, so I'm reviewing my math in order to get up to speed with what I want to do in MatLab. I'm using a couple of self-teaching books, making sure I go from beginning to end, so that I don't drown when doing the programming I need to do. I've stumbled with something simple and feel stupid. I'm given a column vector with imaginary numbers; I'm learning to use the dot product for a matrix with imaginary numbers. I am really, really rusty with respect to "i" and using "i." I know i^2=-1
No sweat in MatLab getting the same answer as in the guides I am learning from; however, when try to multiply the matrix by itself, I cannot get the same answer (I can't get rid of i). So here's the deal:
u=[-i; 1+i; 4+4*i]
dot(u,u)
ans = 35 - so says the text and my work in MatLab
but, I I just try to multiply this out myself to check the work, I get -1 + 32i
(-i*(-i)) + (1+i)*(1+i) + (4+4*i)*(4+4*i) =
-1 +(1+2i -1) +(16 + 32i -16) =
-1 + 32i
What the devil am I doing wrong? Why won't my simple multiplication match the dot(u,u)?
many thanks
Linda

Subject: dot product and vectors with imaginary numbers

From: Matt Fig

Date: 7 Jul, 2010 00:11:04

Message: 2 of 18

In steps:

A = [-i;1+i;4+4i]
Ac = conj(A)
P = A.*Ac
dot(A,A)==sum(P)

Subject: dot product and vectors with imaginary numbers

From: Roger Stafford

Date: 7 Jul, 2010 00:39:08

Message: 3 of 18

"Linda Davis" <linda.l.davis.removethis@jpl.nasa.gov> wrote in message <i10g08$ei8$1@fred.mathworks.com>...
> Ok, so I'm reviewing my math in order to get up to speed with what I want to do in MatLab. I'm using a couple of self-teaching books, making sure I go from beginning to end, so that I don't drown when doing the programming I need to do. I've stumbled with something simple and feel stupid. I'm given a column vector with imaginary numbers; I'm learning to use the dot product for a matrix with imaginary numbers. I am really, really rusty with respect to "i" and using "i." I know i^2=-1
> No sweat in MatLab getting the same answer as in the guides I am learning from; however, when try to multiply the matrix by itself, I cannot get the same answer (I can't get rid of i). So here's the deal:
> u=[-i; 1+i; 4+4*i]
> dot(u,u)
> ans = 35 - so says the text and my work in MatLab
> but, I I just try to multiply this out myself to check the work, I get -1 + 32i
> (-i*(-i)) + (1+i)*(1+i) + (4+4*i)*(4+4*i) =
> -1 +(1+2i -1) +(16 + 32i -16) =
> -1 + 32i
> What the devil am I doing wrong? Why won't my simple multiplication match the dot(u,u)?
> many thanks
> Linda
- - - - - - -
  Check out this Wikipedia website on inner product vector spaces:

 http://en.wikipedia.org/wiki/Inner_product_space

In Euclidean vector spaces over the complex field, the dot product, which is a special kind of inner product, is always defined in terms of complex conjugates as Matt has indicated.

Roger Stafford

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 01:46:06

Message: 4 of 18

"Matt Fig" <spamanon@yahoo.com> wrote in message <i10gmo$t6q$1@fred.mathworks.com>...
> In steps:
>
> A = [-i;1+i;4+4i]
> Ac = conj(A)
> P = A.*Ac
> dot(A,A)==sum(P)

For the symmetric case you can put the conjugate on either argument and get the same result, but since the comparison is to dot a better example would have been:

 P = Ac.*A

since the dot product conjugates the first argument, not the second.

James Tursa

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 01:52:07

Message: 5 of 18

"Linda Davis" <linda.l.davis.removethis@jpl.nasa.gov> wrote in message <i10g08$ei8$1@fred.mathworks.com>...
> dot(u,u)

P.S. The dot function in MATLAB is slow for simple dot product calculations. In general you are better off using straight matrix multiplication or my mtimesx tool on the FEX. e.g., for a column vector u:

u'*u % is faster and more accurate than dot(u,u)
mtimesx(u,'c',u,'speed') % is also faster and more accurate than dot(u,u)

James Tursa

Subject: dot product and vectors with imaginary numbers

From: Linda Davis

Date: 7 Jul, 2010 02:18:05

Message: 6 of 18

Many thanks: this will help when I get to the really huge database.

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i10mk7$dr0$1@fred.mathworks.com>...
> "Linda Davis" <linda.l.davis.removethis@jpl.nasa.gov> wrote in message <i10g08$ei8$1@fred.mathworks.com>...
> > dot(u,u)
>
> P.S. The dot function in MATLAB is slow for simple dot product calculations. In general you are better off using straight matrix multiplication or my mtimesx tool on the FEX. e.g., for a column vector u:
>
> u'*u % is faster and more accurate than dot(u,u)
> mtimesx(u,'c',u,'speed') % is also faster and more accurate than dot(u,u)
>
> James Tursa

Subject: dot product and vectors with imaginary numbers

From: Linda Davis

Date: 7 Jul, 2010 02:20:23

Message: 7 of 18

thank you


"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <i10ibc$cpk$1@fred.mathworks.com>...
> "Linda Davis" <linda.l.davis.removethis@jpl.nasa.gov> wrote in message <i10g08$ei8$1@fred.mathworks.com>...
> > Ok, so I'm reviewing my math in order to get up to speed with what I want to do in MatLab. I'm using a couple of self-teaching books, making sure I go from beginning to end, so that I don't drown when doing the programming I need to do. I've stumbled with something simple and feel stupid. I'm given a column vector with imaginary numbers; I'm learning to use the dot product for a matrix with imaginary numbers. I am really, really rusty with respect to "i" and using "i." I know i^2=-1
> > No sweat in MatLab getting the same answer as in the guides I am learning from; however, when try to multiply the matrix by itself, I cannot get the same answer (I can't get rid of i). So here's the deal:
> > u=[-i; 1+i; 4+4*i]
> > dot(u,u)
> > ans = 35 - so says the text and my work in MatLab
> > but, I I just try to multiply this out myself to check the work, I get -1 + 32i
> > (-i*(-i)) + (1+i)*(1+i) + (4+4*i)*(4+4*i) =
> > -1 +(1+2i -1) +(16 + 32i -16) =
> > -1 + 32i
> > What the devil am I doing wrong? Why won't my simple multiplication match the dot(u,u)?
> > many thanks
> > Linda
> - - - - - - -
> Check out this Wikipedia website on inner product vector spaces:
>
> http://en.wikipedia.org/wiki/Inner_product_space
>
> In Euclidean vector spaces over the complex field, the dot product, which is a special kind of inner product, is always defined in terms of complex conjugates as Matt has indicated.
>
> Roger Stafford

Subject: dot product and vectors with imaginary numbers

From: Linda Davis

Date: 7 Jul, 2010 02:24:22

Message: 8 of 18

thank you Matt -

"Matt Fig" <spamanon@yahoo.com> wrote in message <i10gmo$t6q$1@fred.mathworks.com>...
> In steps:
>
> A = [-i;1+i;4+4i]
> Ac = conj(A)
> P = A.*Ac
> dot(A,A)==sum(P)

Subject: dot product and vectors with imaginary numbers

From: Matt Fig

Date: 7 Jul, 2010 04:49:04

Message: 9 of 18

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i10m8u$lc0$1@fred.mathworks.com>...
> "Matt Fig" <spamanon@yahoo.com> wrote in message <i10gmo$t6q$1@fred.mathworks.com>...
> > In steps:
> >
> > A = [-i;1+i;4+4i]
> > Ac = conj(A)
> > P = A.*Ac
> > dot(A,A)==sum(P)
>
> For the symmetric case you can put the conjugate on either argument and get the same result, but since the comparison is to dot a better example would have been:
>
> P = Ac.*A
>
> since the dot product conjugates the first argument, not the second.
>
> James Tursa


Hmmmm, I was thinking this was a Hermitian Form, as shown here (equation 2):

http://mathworld.wolfram.com/HermitianForm.html

I was taught that the second vector is conjugated in this case, so the result is the conjugate of the MATLAB definition. Is it just convention or what? I realize this is not my area of expertise, so feel free to correct me if I have this wrong....

Subject: dot product and vectors with imaginary numbers

From: Bruno Luong

Date: 7 Jul, 2010 05:22:03

Message: 10 of 18

"Matt Fig" <spamanon@yahoo.com> wrote in message <i11100$3vc$1@fred.mathworks.com>...

>
>
> Hmmmm, I was thinking this was a Hermitian Form, as shown here (equation 2):
>
> http://mathworld.wolfram.com/HermitianForm.html
>
> I was taught that the second vector is conjugated in this case, so the result is the conjugate of the MATLAB definition. Is it just convention or what?

I don't believe there is a universal standard for fixing the first or second argument that being conjugate in the dot product, but yeah in most "classical" textbooks I read (Brezis, Rudin, Ciarley), they often gives

<a,b> = sum(a.*conj(b))

as definition. Matlab DOT function is the opposite.

Both conventions are correct dot products so as to induce an Hilbert space as defined in a more abstract way.
 
Bruno

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 06:41:03

Message: 11 of 18

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <i112tr$9df$1@fred.mathworks.com>...
> "Matt Fig" <spamanon@yahoo.com> wrote in message <i11100$3vc$1@fred.mathworks.com>...
>
> >
> >
> > Hmmmm, I was thinking this was a Hermitian Form, as shown here (equation 2):
> >
> > http://mathworld.wolfram.com/HermitianForm.html
> >
> > I was taught that the second vector is conjugated in this case, so the result is the conjugate of the MATLAB definition. Is it just convention or what?
>
> I don't believe there is a universal standard for fixing the first or second argument that being conjugate in the dot product, but yeah in most "classical" textbooks I read (Brezis, Rudin, Ciarley), they often gives
>
> <a,b> = sum(a.*conj(b))
>
> as definition. Matlab DOT function is the opposite.
>
> Both conventions are correct dot products so as to induce an Hilbert space as defined in a more abstract way.
>
> Bruno

FYI, both Fortran and the BLAS routines conjugate the first, like MATLAB.

James Tursa

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 06:57:09

Message: 12 of 18

"Matt Fig" <spamanon@yahoo.com> wrote in message <i11100$3vc$1@fred.mathworks.com>...

>
> Hmmmm, I was thinking this was a Hermitian Form, as shown here (equation 2):
>
> http://mathworld.wolfram.com/HermitianForm.html

On the same site the question of where the conjugate is applied for the inner product is left ambiguous in this page:

http://mathworld.wolfram.com/InnerProduct.html

e.g., equations (3) and (4).

James Tursa

Subject: dot product and vectors with imaginary numbers

From: Roger Stafford

Date: 7 Jul, 2010 07:22:06

Message: 13 of 18

"Matt Fig" <spamanon@yahoo.com> wrote in message <i11100$3vc$1@fred.mathworks.com>...
> Hmmmm, I was thinking this was a Hermitian Form, as shown here (equation 2):
>
> http://mathworld.wolfram.com/HermitianForm.html
>
> I was taught that the second vector is conjugated in this case, so the result is the conjugate of the MATLAB definition. Is it just convention or what? I realize this is not my area of expertise, so feel free to correct me if I have this wrong....
- - - - - - - - -
  Yes, I was taught the way you were, Matt. A general property that is required of all inner products is that they be linear in the *first* argument, <x*A,B> = x*<A,B>, for any scalar x and vectors A and B. This forces the conjugate to be taken on the second argument B in Euclidean vector space over the complex number field. Moreover dot products, at least in the mathematical world, are generally considered as special kinds of inner products. See these:

 http://en.wikipedia.org/wiki/Dot_product
 http://mathworld.wolfram.com/InnerProduct.html

  I don't know why matlab's dot product applies the conjugate to its first argument unless it was a (misguided?) choice made early on by other computer languages. It is not in agreement with the concept of inner products in the mathematical world.

  It is interesting to note that in the early version of Matlab that I owned, v4a, their dot product there didn't take the conjugate of either argument, so you could for example get a zero result for the dot product of a definitely non-zero complex-valued vector by itself:

 a = [2+i;1-2i];
 dot(a,a) = (2+i)*(2+i)+(1-2i)*(1-2i) = 0

Roger Stafford

Subject: dot product and vectors with imaginary numbers

From: Roger Stafford

Date: 7 Jul, 2010 07:59:05

Message: 14 of 18

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i118g5$og4$1@fred.mathworks.com>...
> On the same site the question of where the conjugate is applied for the inner product is left ambiguous in this page:
>
> http://mathworld.wolfram.com/InnerProduct.html
>
> e.g., equations (3) and (4).
>
> James Tursa
- - - - - - - - -
  James, in point of fact I don't think they are actually ambiguous in the article you refer to. They state there, "With this property (eq. 4), the inner product is called a Hermitian inner product and a complex vector space with a Hermitian inner product is called a Hermitian inner product space." Upon looking up their link to "Hermitian inner product" which is at

 http://mathworld.wolfram.com/HermitianInnerProduct.html

it is clearly stated that:

 3. <alpha u,v> = alpha <u,v>
 4. <u,alpha v> = conj(alpha) <u,v>

are required properties of the Hermitian inner product on a complex vector space. That puts the conjugate operation definitely on the second argument of an inner product.

Roger Stafford

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 08:08:04

Message: 15 of 18

"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <i11c49$j89$1@fred.mathworks.com>...
> "James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i118g5$og4$1@fred.mathworks.com>...
> > On the same site the question of where the conjugate is applied for the inner product is left ambiguous in this page:
> >
> > http://mathworld.wolfram.com/InnerProduct.html
> >
> > e.g., equations (3) and (4).
> >
> > James Tursa
> - - - - - - - - -
> James, in point of fact I don't think they are actually ambiguous in the article you refer to. They state there, "With this property (eq. 4), the inner product is called a Hermitian inner product and a complex vector space with a Hermitian inner product is called a Hermitian inner product space." Upon looking up their link to "Hermitian inner product" which is at
>
> http://mathworld.wolfram.com/HermitianInnerProduct.html
>
> it is clearly stated that:
>
> 3. <alpha u,v> = alpha <u,v>
> 4. <u,alpha v> = conj(alpha) <u,v>
>
> are required properties of the Hermitian inner product on a complex vector space. That puts the conjugate operation definitely on the second argument of an inner product.
>
> Roger Stafford

You're right ... it does indeed make that link to the HeritianInnerProduct page where the conjugate is on the second. I stand corrected. Of course one could come up with a different set of consistent rules with the conjugate on the first as Bruno pointed out earlier. Which set of rules came first?

James Tursa

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 08:17:05

Message: 16 of 18

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i117hv$pm4$1@fred.mathworks.com>...
>
> FYI, both Fortran and the BLAS routines conjugate the first, like MATLAB.

And another "conjugate the first" site:

http://people.ccmr.cornell.edu/~muchomas/8.04/1997/ps6/node3.html

James Tursa

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 08:27:05

Message: 17 of 18

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i11d60$rqk$1@fred.mathworks.com>...
> "James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i117hv$pm4$1@fred.mathworks.com>...
> >
> > FYI, both Fortran and the BLAS routines conjugate the first, like MATLAB.
>
> And another "conjugate the first" site:
>
> http://people.ccmr.cornell.edu/~muchomas/8.04/1997/ps6/node3.html

And another one:

http://web.mit.edu/18.06/www/Fall07/operators.pdf

If you think of vectors as column vectors, then the dot product works out nicely as x^H * x, rather than writing x^T * conj(x), as the last article points out. Maybe this had something to do with why Fortran (and the follow on BLAS interface and MATLAB) chose to conjugate the first instead of the second. I don't really know. Maybe a post to the Fortran newsgroup is in order.

James Tursa

Subject: dot product and vectors with imaginary numbers

From: James Tursa

Date: 7 Jul, 2010 08:37:06

Message: 18 of 18

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i11dop$510$1@fred.mathworks.com>...
> "James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i11d60$rqk$1@fred.mathworks.com>...
> > "James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <i117hv$pm4$1@fred.mathworks.com>...
> > >
> > > FYI, both Fortran and the BLAS routines conjugate the first, like MATLAB.
> >
> > And another "conjugate the first" site:
> >
> > http://people.ccmr.cornell.edu/~muchomas/8.04/1997/ps6/node3.html
>
> And another one:
>
> http://web.mit.edu/18.06/www/Fall07/operators.pdf
>
> If you think of vectors as column vectors, then the dot product works out nicely as x^H * x, rather than writing x^T * conj(x), as the last article points out. Maybe this had something to do with why Fortran (and the follow on BLAS interface and MATLAB) chose to conjugate the first instead of the second. I don't really know. Maybe a post to the Fortran newsgroup is in order.

And it also appears that one of the technology labs at Indiana University is defining C++ vector operations code that has first conjugated:

http://osl.iu.edu/research/mtl/mtl4/doc/namespacemtl_1_1vector.html#_details

So it appears that the many textbooks and online "math" sites have the second conjugated, while computer languages have the first conjugated (maybe everybody is just following Fortran). I would be curious to know if there are any computer languages that have the second conjugated.

James Tursa

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us