Asked by Xiaohan Du
on 26 Jun 2018

Hi all,

Imagine I have 2 large matrices which have more rows than columns, I'd like to calculate trace(A' * B) for N times. I have 2 options: 1. calculate trace(A' * B) directly; 2. only calculate vector product of the diagonal, then sum it. I test with the following minimum example, it turns out the 2nd option is faster:

clear; clc;

nd = 100000;

nt = 500;

A = rand(nd, nt);

B = rand(nd, nt);

N = 100;

%%no SVD.

% option 1.

tic

for i = 1:N

abTr = trace(A' * B);

end

toc

% option 2.

tic

abSum = 0;

for i = 1:N

for j = 1:nt

abSum = abSum + A(:, j)' * B(:, j);

end

end

toc

Elapsed time is 58.320345 seconds.

Elapsed time is 13.558039 seconds.

Now I need to apply truncated-SVD to A and B to optimise storage. the following code is applied to leave only 10 vectors

% apply svd

[ua, sa, va] = svd(A, 'econ');

[ub, sb, vb] = svd(B, 'econ');

% group in cells

nRem = 10;

ucell = {ua va ub vb};

scell = {sa sb};

% truncate

ucell = cellfun(@(v) v(:, 1:nRem), ucell, 'un', 0);

scell = cellfun(@(v) v(1:nRem, 1:nRem), scell, 'un', 0);

My question is: is it possible to calculate trace (it's an approximation due to truncation) using option 2 with these SVD vectors in ucell and singular values in scell? I can do it with option 1 using

trace((vb' * va * sa' * (ua' * ub) * sb)

But I'd like to be faster by only considering diagonal elements.

Many thanks!

Answer by Anton Semechko
on 26 Jun 2018

Edited by Anton Semechko
on 26 Jun 2018

Accepted Answer

I am not aware of any identities that can express trace(A'*B) in terms of SVDs of A and B, and I dont think one exists, but there is an identity that does not require explicit calculation of the A'*B matrix:

trace(A'*B) = sum(A(:).*B(:))

For example,

nd = 100000;

nt = 500;

A = rand(nd, nt);

B = rand(nd, nt);

N = 100;

tic

tr1=trace(A'*B);

t1=toc;

%Elapsed time is 0.857053 seconds.

tic

t2=sum(A(:).*B(:));

toc

%Elapsed time is 0.210807 seconds.

As you can see, you obtain the result about 4 times faster when using the above identity. This identity is especially useful when nt is 'large', because the machine may not have insufficient memory to hold the nt-by-nt matrix A'*B.

Anton Semechko
on 28 Jun 2018

Xiaohan Du
on 28 Jun 2018

trace(A' * B) = sum(diag(A' * B)), so actually diagonal elements are needed rather than all elements right?

My understanding is: in order to calculate diagonal elements of (A' * B), only paired column vectors of A and B are needed, i.e. if A = [ai]_i = 1^n, B = [bi]_i = 1^n, then only ai' * bi needs to be calculated.

However, this calculation of diagonal elements only cannot be achieved once A and B are decomposed by SVD, because recovering A' * B by SVD vectors recovers a full matrix which contains part of the entire diagonal line, rather than one or a few elements on the diagonal line.

Anton Semechko
on 28 Jun 2018

I thought the problem was resolved because you found an efficient way to evaluate tr(A'*B) using SVDs of A and B.

Nevertheless, of course it is possible to recover ONLY the diagonal elements of A'*B from the SVDs of A and B while staying within memory constraints of the machine. However, I am not sure that computing these diagonal elements first and then summing them would help you improve evaluation of tr(A'*B).

Sign in to comment.

Answer by Jan
on 26 Jun 2018

Edited by Jan
on 26 Jun 2018

I assume this is faster:

nd = 100000;

nt = 500;

A = rand(nd, nt);

B = rand(nd, nt);

tic;

abTr = trace(A' * B);

toc % Elapsed time is 0.497233 seconds.

tic;

abSum = 0;

for j = 1:nt

abSum = abSum + A(:, j)' * B(:, j);

end

toc; % Elapsed time is 0.141059 seconds.

tic

abTr2 = sum(A(:).*B(:));

toc % Elapsed time is 0.182408 seconds.

tic;

abTr3 = A(:).' * B(:);

toc % Elapsed time is 0.054548 seconds.

tic;

[ua, sa, va] = svd(A, 'econ');

[ub, sb, vb] = svd(B, 'econ');

toc % Elapsed time is 10.629806 seconds.

% And this does not even include the further computations

% relative error:

abs(abTr - abTr3) / abTr

The reshaping by (:) and the transposition creates at least shared data copies, such that no additional memory is used. Maybe the JIT acceleration can call the dot product of the BLAS library directly, where the transposition can be provided as a flag.

Of course for a more accurate measurement some loops would be smarter, or better use timeit. But a factor of 3 ( abTr3 against abSum) is significant enough.

Xiaohan Du
on 27 Jun 2018

Xiaohan Du
on 29 Jun 2018

Hi Jan,

I don't understand why

A(:).' * B(:)

is faster than

for j = 1:nt

abSum = abSum + A(:, j)' * B(:, j);

end

theoretically, the second operation only has nt operations, each is a vector product between the ith vectors of A and B, while the first has to calculate the product between every 2 vectors in A and B, so shouldn't the first cost more than the second?

Jan
on 29 Jun 2018

A(:) creates a shared data copy: Only the header of the variable is changed, but the data are not copied. The same happens for the transposition. But Matlab's JIT acceleration might be so smart and avoid even the shared data copies by calling the underlying BLAS function DDOT directly. This function is implemented with multi-threading - you can see this in the task-manager, where all cores are loaded during the processing.

So A(:).' * B(:) starts as many threads as cores are available and no memory has to be copied around in the slow RAM.

abSum = abSum + A(:, j)' * B(:, j) calls DDOT also and starts multiple threads. The starting of a thread is very costly and here we need 4 starts for each column. I'm not sure if Matlab's JIT is smart enough to avoid the creation of the temporary vectors A(:,j) and B(:,j) in each iteration. But if this is performed explicitly, two variables have to be created and the elements of the columns are copied. Calling the library function DDOT costs some time for the overhead also.

Summary:

- Case 1: DDOT is called one time and 4 threads are started.
- Case 2: For nt=500, 1000 columns are copied, DDOT is called 500 times, 2000 threads are started (on a 4 core CPU).

So for me the speedup is expected.

the first has to calculate the product between every 2 vectors in A and B

This might be a misunderstanding: A(:) and B(:) are two vectors only, not "every two vectors".

Sign in to comment.

Opportunities for recent engineering grads.

Apply Today
## 0 Comments

Sign in to comment.