convert for loop into parfor loop
Show older comments
Can somebody point out how I have to modify the code below to allow it being used in a parfor loop
H = height(RTImport)
location = 'location..'
ds = datastore(location)
TT = tall(ds)
func = @(x) mean(x,'omitnan');
x1{H} = {};
x2{H} = {};
tic
for i = 1:H
Strt = minutes(RTImport.Start(i))
endT = Strt + minutes(1)
S = timerange(Strt,endT,'closed')
TT1 = TT(S,:)
Strt = Strt + minutes(1)
endT = Strt + minutes(1)
S = timerange(Strt,endT,'closed')
TT2 = TT(S,:)
x1{i} = varfun(func,TT1,'OutputFormat','table')
x2{i} = varfun(func,TT2,'OutputFormat','table')
gather(x1{:},x2{:})
end
toc
I get the information unable to classify x1 in the body of the parfor. Going through the information, I couldnt really figure out what wrong with it...Appreciate any help
6 Comments
Rik
on 17 Nov 2020
Did you try pre-allocating it to the correct size? And why aren't you storing the result of gather in a variable? And why are you only using 3 semicolons? There is no reason to omit them if you are already at the stage that you worry enough about the performance that you want to use parfor.
Lutetium
on 17 Nov 2020
Rik
on 17 Nov 2020
I prefer a more explicit pre-allocation like this:
x1=cell(1,H);
Your call to gather doesn't actually do anything here. It has an output, but that isn't stored, so your x1 variable remains unchanged. It is very rare in Matlab that a call to a function will modify the input in the calling workspace. I don't know of any examples outside the symbolic toolbox with functions like assume. In short: your call doesn't do anything, so it can be omitted. If you need it, you should probably use this instead:
[x1{i},x2{1}]=gather(x1{i},x2{i});
(but I am not sure why you would need it)
A small note: because you're using a parallel loop you can't be sure about the order, and no iteration can depend on any other (so the concept of needing to gather variables before doing the next iteration may be flawed). For some calculations it takes more time to copy the variables to all the separate workers that to do the calculation in a single thread. This overhead is the reason why parfor is in some cases slower than for.
Lutetium
on 17 Nov 2020
Lutetium
on 17 Nov 2020
Rik
on 17 Nov 2020
You're welcome.
Answers (0)
Categories
Find more on Loops and Conditional Statements in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!