using parfeval for asynchronous parallel function evaluation

Hi everyone,

I try to parallel function evaluation on workers. Suppose I have 10 different vectors that will be evaluated through same function. I want to retrieve outputs when they become ready. Also, when the certain number of outputs is reached, I want to evaluate the retrieved outputs through a different function without waiting to fetch all of the results. Moreover, when the unretrived data of the agents become ready, I want to use it on the same fc2 function. I wonder is it possible or not? I wrote something like this.

while itr<itrmax
    for j=1:nbagents
        f(j) =parfeval(p,@evaluate,1,agent(j));
    end
    counter=0;
    finishedjobs=[];
    for idx = 1:nbagents
        [completedIdx,value] = fetchNext(f);
        finishedjobs=[finishedjobs completedIdx];
        agent(completedIdx)=value;
        counter=counter+1;
        if counter==5
            break;
        end
    end
    for i=1:counter
        output(i)=agent(finishedjobs(i))
    end
    target=fc2(output(i));
    itr=itr+1;
end

[Edric Ellis]

I'm a little bit confused by your code here. It sounds like you should be able to do what you want. Your current code appears to correctly retrieve the first 5 results to become ready, and overwrites the corresponding elements of agent.

After your fetchNext loop, you have still some incomplete parfeval futures - what do you intend to do with those?

It might be easier to use a logical array to track the completed futures, like so:

isFinished = false(1, numel(f));
for idx = 1:5 % (you could set the loop bounds rather than using 'break')
    [completedIdx, value] = fetchNext(f);
    isFinished(completedIdx) = true;
    ...
end
% Get outputs from all finished futures:
finishedOutputs = fetchOutputs(f(isFinished));
% Here are the incomplete futures
incompleteFutures = f(~isFinished);
% Not sure what you want to do with these...
% you could call 'cancel(incompleteFutures)' to discard?

[Emre Senol]

I want to incomplete parfeval futures to use when they finished on the same fc2 function. Also, It will be a continuing process. Whenever 5 of the retrieved results become ready, they will be used as an input for fc2 function and output of the fc2 function will be input for parfeval function again. I wonder is it possible or not?