accelerating a code via parfor
1 view (last 30 days)
Show older comments
Is there any ways to speed up the code below? It is too slow on my computer? When I type in parpool, is that enough to use all the cores of my i5 CPU for calculating this code or must I insert a parfor code into my code below. If so, how do I insert a parfor code in the code below? Thanks
for x = 1:100
for y = 1:61
b = abs(bsxfun(@minus,q{x,y},kevin))
averagesAcrossColumns = mean(b, 2);
b = abs(bsxfun(@minus,b,averagesAcrossColumns));
b = sum(b, 2);
s{x,y} = b;
end
end
5 Comments
Matt J
on 9 Dec 2014
Edited: Matt J
on 9 Dec 2014
Kevin is a 1x60 matrix and q is a 100x 61 cell array.
The memory consumed by a 100x61 cell array, each containing a double precision matrix of size 1e6 x 60 is about 2.6 TB! The only way you could be storing such a matrix on any normal computer is if some of the q{x,y} are shared duplicates of each other. If that's true, you could speed things up by getting rid of the duplicates.
Answers (2)
Matt J
on 8 Dec 2014
Edited: Matt J
on 8 Dec 2014
There is no need to have a double loop. See if the following improves things. If not, tell us more about the contents of each q{x,y}, in particular how many columns it has.
s=cell(size(q));
parfor i=1:numel(q)
b = abs(bsxfun(@minus,q{i},kevin))
averagesAcrossColumns = mean(b, 2);
b = abs(bsxfun(@minus,b,averagesAcrossColumns));
b = sum(b, 2);
s{i} = b;
end
0 Comments
Matt J
on 8 Dec 2014
If all q{x,y} have the same number of columns, you don't need a loop at all,
b = abs(bsxfun(@minus,cat(3,q{:}),kevin))
averagesAcrossColumns = mean(b, 2);
b = abs(bsxfun(@minus,b,averagesAcrossColumns));
b = sum(b, 2);
s=reshape(num2cell(b,1),size(q));
6 Comments
Matt J
on 9 Dec 2014
Edited: Matt J
on 9 Dec 2014
You said in your comment here "Each cell in q consists of matrices with million rows". If that's true, then q must contain a minimum of 1e6*60*100*61 values. In doubles, that is 2.6TB. So, you should correct that comment so we understand what's really going on.
Anyway, you haven't commented on my other Answer (with parfor). You also haven't commented on the possibility of padding all q(x,y) to have the same number of rows.
A GPU solution is starting to sound doubtful. You will need lots of RAM (<5GB) on the card and it will take a long time to transmit the data to the card.
See Also
Categories
Find more on Loops and Conditional Statements in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!