Why is job submit time code dependent?

2 views (last 30 days)
Sean Larkin
Sean Larkin on 8 Aug 2013
I have set up a test program to evaluate the batch job/task capability of the PCT. I set it up to run using this:
parallel.defaultClusterProfile('local');
clust = parcluster();
batchJob = createJob(clust);
createTask(batchJob, @itersum, 2, {1,1});
createTask(batchJob, @itersum, 2, {2,1});
tic;
submit(batchJob);
toc;
wait(batchJob)
batchout = fetchOutputs(batchJob);
My task function just adds one of the inputs to itself a number of times in a loop. There are 5 lines in the code. This takes ~.2 seconds to initialize in the submit function. If I substitute A much more lengthy function for my task function it takes ~10 seconds to initialize. If I comment out some of the code in my task function it takes less time to intialize. Why is this the case? Doesn't the submit function just set up matlab workers and some other settings?
  3 Comments
Sean Larkin
Sean Larkin on 8 Aug 2013
Edited: Sean Larkin on 8 Aug 2013
Yes, the timings are consistent regardless of order. Basically more code leads to longer initialization times.
Sean Larkin
Sean Larkin on 8 Aug 2013
There is a property of parallel.Job called 'AutoAttachFiles'. It seems that a job gets files attached it to other than the task function files. Does anyone know what is meant by 'attached' in regards to a file? When I create my job I can set the 'AutoAttachFiles' property to false and the submit function runs very fast, <.1 seconds. Here's my hypothesis as to why the submit job can take so long. The submit function must scan through the code and find all file/function dependencies, almost like a compiler. Then it attaches or doesn't attach these file, as I haven't figure that out yet. Either way, the scanning of the code must be rather time consuming, thus causing the timings that I reported. This information should be in the help files or some documentation, I could not find it. So, will setting the 'AutoAttachFiles' property to false have any effects on the code running correctly? It doesn't affect my code, but this could be coincidental.

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!