The variable s in a parfor cannot be classified.

2 views (last 30 days)
I would like to run the following code snippet in parallel as the size of the data is too huge.
data_no = 10099;
alpha = 0.5;
m = 1;
parfor i=1:data_no
parfor j=i+1:data_no
s1 = norm(data_corr(i,:)-data_corr(j,:));
s2 = norm(data(i,:)-data(j,:));
d = s1 + alpha*s2;
s(m,1) = i;
s(m,2) = j;
s(m,3) = -d;
m = m+1;
end
end
But I get the error "The variable s in a parfor cannot be classified". I have not had much luck with understanding the matlab help unfortunately.
Can somebody please help.
Thanks,
Janki

Accepted Answer

Matt J
Matt J on 30 Jan 2015
Here is a method that avoids looping altogether. I expect it would be much faster than a parfor approach.
data_no=size(data_corr,1);
[I,J]=ndgrid(1:data_no);
idx=(J>I);
D=normMatrix(data_corr)+alpha*normMatrix(data);
s=[I(idx), J(idx), -D(idx)].';
function S=normMatrix(data)
normterms=sum(data.^2,2);
crossterms=data*data.';
S=bsxfun(@minus,normterms, 2*crossterms);
S=bsxfun(@plus,S,normterms.');
S=sqrt(S);
end
  2 Comments
Janki Mehta
Janki Mehta on 5 Feb 2015
Thanks a lot Matt. The code took about 100 times less time to run.
But I still have one more issue. My data (fMRI time series) is about the size 60000x100, which gives me errors of Out of Memory. I guess this can possibly be solved if use some high-performance workstation.
But the bigger problem is the following error for the line
[I,J]=ndgrid(1:60000);.
Maximum variable size allowed by the program is exceeded.
Error in ndgrid (line 63)
varargout{1} = x(:,ones(size(y)));
Can you please help me this?
Thanks, Janki
Matt J
Matt J on 5 Feb 2015
Edited: Matt J on 5 Feb 2015
With data_no=60000, the 's' you are trying to compute would be 40GB in double floats or half that in single floats. You could get that much RAM if you were really determined, I suppose, but you should probably be rethinking your goals instead...

Sign in to comment.

More Answers (1)

Matt J
Matt J on 30 Jan 2015
Edited: Matt J on 30 Jan 2015
[I,J]=ndgrid(1:data_no);
idx=(J>I);
I=I(idx); J=J(idx);
N=length(J);
s=nan(3,N);
parfor m=1:N
i=I(m);
j=J(m);
s1 = norm(data_corr(i,:)-data_corr(j,:));
s2 = norm(data(i,:)-data(j,:));
d = s1 + alpha*s2;
s(:,m)=[i;j;-d];
end
s=s.';

Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!