Vectorizing a for loop

I need some help vectorizing some code as I know that it's slowing down my run time quite a bit. I have a matrix that is sorted by rows based on one of the columns. I then use the 'unique' function to pull out the unique values of one of the columns (which also returns index vectors). The goal is then to produce average values of another column of the original matrix based on the unique values. Here is the code:

[C,idxa,idxc] = unique(A(:,17));  %pulls out unique values of col 17
C = horzcat(C,zeros(length(C),1)); %extends C by a column
for i=1:length(idxa)
   C(i,2) = mean(A(eq(idxc,i),16));
end;

So you can see that I'm trying to find the mean of all of the rows in col 16 that correspond to the same unique values of col 17. This code gets the job done but it's really, really slow and I know that vectorizing it would help. Here is the code that would generate the first and last means:

C(1,2) = mean(A(idxa(1):idxa(2)-1));
C(end,2) = mean(A(idxa(end):end));

Any help is greatly appreciated!

Answer 1

[Geoff Hayes]

Rob - is the matrix sorted on column 17? And can you quantify what you mean by really, really slow - does it take seconds, minutes, hours (!) to find the result? Are there that many rows in A?

Your final two lines of code

 C(1,2)   = mean(A(idxa(1):idxa(2)-1));
 C(end,2) = mean(A(idxa(end):end));

ignores the 16th column of A. Is this intentional?

Since you want to average the elements of the 16th row of A that correspond to the unique values of the 17th row, then you could try something like the following using arrayfun which applies a function to each element of an input matrix

 % add another element to idea which will be the number of rows of A plus one
 % we use this last element to indicate where we should "stop" in calculating the
 % average/mean for the final unique number
 idxa = [idxa;size(A,1)+1];

 % now use arrayfun to calculate the average for each set of unique values (from the
 % 17th column) that are in the 16th column
 D = arrayfun(@(x)mean(A(idxa(x):idxa(x+1)-1,16)),1:length(idxa)-1)';

The above is very similar to what you coded for the first and last means. We average all data in the 16th column of A for the rows idxa(x):idxa(x+1)-1. Our input to arrayfun is the anonymous function

 @(x)mean(A(idxa(x):idxa(x+1)-1,16))

whose input parameter x is based on the second input to arrayfun

1:length(idxa)-1

which is just a list of indices from 1 to the end of idxa less one (since the last element of idxa is dummy that we added). So if A has 15 rows and idxa is

 idxa =

     1
     3
     4
     5
     7
     8
    11
    15
    16  <---this is the value we added

Then the means are calculated given the pair of row indices (1,2), (3,3), (4,4), (5,6), (7,7), (8,10), (11,14), and (15,15).

[Rob]

Geoff,

You're right, I forgot the 2nd index in the last two lines of code. It should be:

 C(1,2)   = mean(A(idxa(1):idxa(2)-1,16));
 C(end,2) = mean(A(idxa(end):end,16));

Let me read through the rest of your answer to see if I can make that work. Thanks a bunch for your input!

[Rob]

Geoff,

Your suggested code appears to be working great! Thanks very much. I had two of these loops in my code and the run time dropped from about 20 minutes to just a few seconds. It really points to the power of good code. I knew there was a way to do it but I don't use MATLAB enough to remember all of the tricks.

Thank you! Thank you!

[Geoff Hayes]

Great that the above worked out, Rob!