Problem with parfor on distributed computing server
15 views (last 30 days)
Show older comments
I decided to run a simple example to make sure things are working correctly with the MATLAB Distributed Computing Server at my university, but I'm having some problems with it. Specifically, code using a parfor loop that runs correctly in on my local PC does not seem to be running in parallel on the remote server. Instead, it seems that each worker on the server is running every iteration of the parfor loop. The code I'm running is below (my default parallel cluster is set to the distributed computing server):
function RunBatchTest
TestBatch(1,'local')
TestBatch(3,'local')
TestBatch(1,'remote')
TestBatch(3,'remote')
function out = TestBatch(numWorkers,clusterLocation)
fun = @TestParallel;
switch clusterLocation,
case 'local'
cluster = parcluster('local');
case 'remote'
cluster = parcluster;
ClusterInfo.setMemUsage('4000');
ClusterInfo.setWallTime('1:00:00');
case 'cloud'
cluster = parcluster('MATLAB Parallel Cloud');
end
display(['Cluster location: ' clusterLocation]);
display(['Number of workers: ' num2str(numWorkers)]);
job = batch(cluster,fun,2,'Pool',numWorkers,'CurrentDirectory','.');
try
job.wait;
out = fetchOutputs(job);
catch
end
diary(job);
function [t,V] = TestParallel
n = 1000;
numRuns = 60;
t0 = tic;
V = cell(numRuns,1);
parfor i = 1:numRuns,
A = rand(n,n);
[~,D] = eig(A);
V{i} = max(D(:));
end
t = toc(t0);
disp(['Computation time: ' num2str(t)]);
And this is the ouput I get:
>> RunBatchTest
Cluster location: local
Number of workers: 1
--- Start Diary ---
Computation time: 51.0082
--- End Diary ---
ans = [51.0082] {60x1 cell}
Cluster location: local
Number of workers: 3
--- Start Diary ---
Computation time: 27.7038
--- End Diary ---
ans = [27.7038] {60x1 cell}
Cluster location: remote
Number of workers: 1
additionalSubmitArgs =
-l nodes=1:ppn=2 -l walltime=1:00:00 -l pmem=4000mb -W x=GRES:MATLAB_Distrib_Comp_Engine+2
--- Start Diary ---
Computation time: 82.243
Computation time: 82.3664
--- End Diary ---
ans = [82.3664] {60x1 cell}
Cluster location: remote
Number of workers: 3
additionalSubmitArgs =
-l nodes=1:ppn=4 -l walltime=1:00:00 -l pmem=4000mb -W x=GRES:MATLAB_Distrib_Comp_Engine+4
--- Start Diary ---
Computation time: 85.146
Computation time: 85.6388
Computation time: 85.9853
--- End Diary ---
ans = [85.9853] {60x1 cell}
>>
As you can see, the local parallel cluster performs as expected for both 1 and 3 workers, but the remote cluster does not. The fact that line containing the text "Computation time:" is displayed to the console once for each worker indicates to me that each worker is running the entire function. Any ideas on how to fix this?
0 Comments
Answers (0)
See Also
Categories
Find more on MATLAB Parallel Server in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!