Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
parfor - unnecessary communication overhead

Subject: parfor - unnecessary communication overhead

From: Jveer

Date: 1 Feb, 2009 14:42:01

Message: 1 of 7

i get the following when using a parfor loop

'variable indexed but not sliced,in a PARFOR loop. this might result in unnecessary communication overhead.

i havent got a clue what do with this. does anyone know how to resolve this?

Subject: parfor - unnecessary communication overhead

From: Edric M Ellis

Date: 2 Feb, 2009 08:16:34

Message: 2 of 7

"Jveer " <jveer@jveer.com> writes:
> i get the following when using a parfor loop
>
> 'variable indexed but not sliced,in a PARFOR loop. this might result in unnecessary communication overhead.
>
> i havent got a clue what do with this. does anyone know how to resolve this?

A simple example of an "indexed but not sliced" variable is "a" here:

a = rand(100);
b = rand(100);
c = zeros(1,10);
parfor ii=1:10
    aidx = randi( [1 100], 1, 2 );
    c(ii) = a( aidx(1), aidx(2) ) + b(1,ii);
end

whereas "b" is sliced.

A "sliced" variable is one which is indexed in such a way that only a portion of
the full value needs to be sent to the worker responsible for evaluating a given
range of iterations of the loop.

Consider the variable "b" in the loop above. Let's imagine that you have a
MATLABPOOL open of size 4. What will happen is that each worker gets a chunk of
the index range to evaluate - let's say a worker gets the range ii =
[8:10]. From the way you've indexed into "b", it's clear that that worker only
needs the portion "b(:,8:10)" which is much smaller than the full value of "b".

However, because "a" is indexed in a way which does not depend on the loop
variable, the full value will be sent to the workers prior to executing the
loop. In the example above, I've deliberately made it clear that one cannot
predict which bits of "a" are needed. However, even a simpler reference to "a"
that does not involve the loop variable causes "a" not to be sliced. For
example, the full value of "a" would still be sent if the body was:

    c(ii) = a(2, 3) + b(1,ii);

In cases like that, you can work around this by sub-selecting "a" outside the
loop, like this:

a23 = a(2,3)
parfor ii=1:10
    c(ii) = a23 + b(1,ii);
end

Cheers,

Edric.

Subject: parfor - unnecessary communication overhead

From: Jveer

Date: 2 Feb, 2009 10:23:02

Message: 3 of 7

thanks a lot. that's very useful information.

however i cannot see how i could possibly slice the variables in this code. any suggestions? r,s and P are not sliced.

something else : do you if its possible to use global variables in parfor loops?


load a.mat P NCx NCy NCz

Ux=max(NCx,[],2)';Lx=min(NCx,[],2)';
[t,r]=sort([Lx,Ux,P(1,:)]);s=1:length(r);s(r)=s;

% Finding all Material Points that could be in cell c using the x, y and z
% limits of the cell
m=length(NCx(:,1));
parfor c=1:m% For each cell
    nx=r(s(c):s(c+m));nx=nx(nx>2*m)-2*m;
    if length(nx);
        ny=nx(P(2,nx)>=min(NCy(c,:)));
        if length(ny)
            ny=ny(P(2,ny)<max(NCy(c,:)));
            if length(ny);
                nz=ny(P(3,ny)>=min(NCz(c,:)));
                if length(nz)
                    nz=nz(P(3,nz)<max(NCz(c,:)));
                    if length(nz);
                        PiC_P{c}=nz;
                    end
                end
            end
        end
    end
end

L=length(P(1,:));k=0;
for c=1:length(PiC_P)
    if length(PiC_P{c})
        k=k+1;
        PiC{1}(k)=c;
        if L<=127,PiC{2}{k}=int8(PiC_P{c});
        elseif L<=32767,PiC{2}{k}=int16(PiC_P{c});
        elseif L<=2147483647,PiC{2}{k}=int32(PiC_P{c});
        else PiC{2}{k}=int64(PiC_P{c});
        end
    end
end

Subject: parfor - unnecessary communication overhead

From: Edric M Ellis

Date: 2 Feb, 2009 11:20:13

Message: 4 of 7

"Jveer " <jveer@jveer.com> writes:

> thanks a lot. that's very useful information.
>
> however i cannot see how i could possibly slice the variables in this
> code. any suggestions? r,s and P are not sliced.

It is not required that you slice input variables - it's just that if they are
large, more data transmission will occur. How large (in MB) are r, s, P?

The first expression in the loop:

nx=r(s(c):s(c+m))

prevents both s and r from being sliced as they are both indexed in a way that
is not a simple combination of constant indices and the loop variable. To have
sliced inputs, you need to have the situation where the inputs naturally divide
corresponding to the loop variable.

> something else : do you if its possible to use global variables in parfor loops?

The "global" statement is illegal within the body of a parfor loop; however, you
may call functions that use "global". If you do that, you must realise that each
worker has its own version of the "global" variable - there is no magic to
ensure that global/persistent data is synchronised across workers. We only send
over those variables that are "transparently" sent to the body of the loop.

Cheers,

Edric.

Subject: parfor - unnecessary communication overhead

From: Jveer

Date: 2 Feb, 2009 21:22:01

Message: 5 of 7

that makes sense. thank you Edric

from the code posted above, is there any way to avoid using the second for loop to construct the required variable? is it possible to somehow workaround the limitations of indices within the parfor loop to get PiC directly?

Subject: parfor - unnecessary communication overhead

From: Edric M Ellis

Date: 3 Feb, 2009 08:30:29

Message: 6 of 7

"Jveer " <jveer@jveer.com> writes:

> from the code posted above, is there any way to avoid using the second for
> loop to construct the required variable? is it possible to somehow workaround
> the limitations of indices within the parfor loop to get PiC directly?

I think you might be able to do this using concatenation rather than indexing to
produce the output. For example:

xVals = {};
idxs = {};
parfor ii=1:10
    x = rand;
    if x > 0.5;
        xVals = [ xVals, {x} ];
        idxs = [ idxs, {ii} ];
    end
end

You'll need to experiment to see whether that is more or less efficient for your
situation.

Cheers,

Edric.

Subject: parfor - unnecessary communication overhead

From: Jveer

Date: 6 Feb, 2009 00:45:04

Message: 7 of 7

i had tried that before. sometimes it's slower, sometimes it takes the same amount of time. i think parfor runs at its best if the indices are set as x(ii) rather than concatenating.

i was wondering whether its more efficient to emlmex the functions with for loops rather than using parfor? however it appears that parfor doesnt work when compiled.

given how much faster for loops are in compiled applications, do u happen to know how the compiled application can be made to split those loops over a cluster of networked computers?

thank you for all the help Edric

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us