Array changes size when passed to a reduction function in the parfor loop?

1 view (last 30 days)
I ran into a weird problem when I used numerical arrays or structure arrays inside a parfor loop. Below is my main function:
comm_struct=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
a(1:100,1)=comm_struct;
b(1:10,1)=comm_struct;
disp(size(a))
disp(size(b))
parfor ct=1:12
b=max_select_2(b,a);
end
where max_select_2 is
function y=max_select_2(b,a)
if size(b,1)~=10||size(a,1)~=100
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d(1:110,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:10,1)=b;
d(11:110,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:10));
If I replace the parfor with for and run on my local matlab, no problem, but if I use parfor and run the code on a cluster, disp(size(a)) and disp(size(b)) generates output
100 1
10 1
which is expected but I also got the error message
"size of a is 100,size of b is 100"
which means for some reason when b is passed to the function max_select_2, its size changed from 10 to 100? I doubt that it has something to do with how matlab handles the structure array but I don't really know what's going on and how to fix it...
For further information, I tweaked the code to help to diagnose.
If I change a(1:100,1)=comm_struct to a(1:10,1)=comm_struct;, and max_select_2 becomes
function y=max_select_2(b,a)
disp(size(b,1));
d(1:20,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:10,1)=b;
d(11:20,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:10));
the output from disp(size(a)) and disp(size(b)) in the main body are
10 1
10 1
and disp(size(b,1)) in max_select_2 generates 10 which is the expected result!
If I set the size of b to 100 and size of a to 10 in the main function and max_select_2 becomes
function y=max_select_2(b,a)
if size(b,1)~=100||size(a,1)~=10
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d(1:110,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:100,1)=b;
d(101:110,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:100));
the outputs from disp(size(a)) and disp(size(b)) in the main function are
10 1
100 1
But I get the error message
size of a is 10,size of b is 10
If I modify the parfor loop in the main function as
parfor ct=1:12
c=max_select_2(b,a);
end
so that b is no longer a reduction variable, I don't get the error message anymore.
So apparently the size of the structure array b changes to that of a when b is a reduction variable. Is that expected? Could it be a bug?
Also, I had the same problem even if a and b are just regular arrays. i.e., I got the error message
size of a is 10,size of b is 10
when the main function is
a=zeros(10,1);
b=zeros(100,1);
disp(size(a));
disp(size(b));
parfor ct=1:12
b=max_select_array(b,a);
end
and max_select_array is
function y=max_select_array(b,a)
if size(b,1)~=100||size(a,1)~=10
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d=zeros(110,1);
d(1:100,1)=b;
d(101:110,1)=a;
[~,sort_idx]=sort(d,'descend');
y=d(sort_idx(1:100));
Anybody has encountered similar problems? I'm using Matlab 2015a. Thanks!
Li

Answers (0)

Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!