[Optimization Tool] Parallelization not working
5 views (last 30 days)
Show older comments
I have a problem using the Optimization Tool with parallel evaluations of the fitness functions.
My fitness function starts an external program using the 'system(external program.exe -par1 - par2 .....)'. This external program calculates the fitness and write the fitness value in a text file. The fitness function reads this value and returns it to Matlab. Everything is working smoothely when i use the option "in serial" at "evaluate fitness and constraint functions".
Now I want to run the fitness program several times parallel. So I use >> matlabpool open local 4 and switch "Evaluate fitness and constraint functions" to "in parallel".
After I press start, I can see in the task manager that my external programm is getting started 4 times but only for 2 seconds. Then it gets killed. There are no error messages nor a window of the external program.
My guess is that Matlab is opening the external program in a different way. Any idea to avoid this or to solve this problem
Thanks! Andi
2 Comments
Answers (3)
Titus Edelhofer
on 30 Aug 2012
Hi,
I suspect the problem on the external program side: you might try to start manually the external program twice. E.g. can you tell your external program to use a different output file name for each loop? I.e., something like
outFile = ['result_' num2str(i)];
because otherwise all 4 instances of the program try to write to the same file (which will fail) ...
Titus
Jason Ross
on 30 Aug 2012
If you get the return values of system(), is there anything interesting in there?
[status, result] = system('command')
Alan Weiss
on 30 Aug 2012
Edited: Alan Weiss
on 31 Aug 2012
The documentation states that accessing external files is problematic when using parallel computation.
I suppose that your fitness function computation is by far the costliest part of the calculation, and yet it seems you have only one computer performing the fitness calculation. So I do not see how you expect parallel computation to be beneficial. If each core could send its computation to a separate computer, and if you could get the results without writing files, then you might get speedup by computing in parallel. As it is, I am afraid that you are not going to see speedup, even if you could get the parallel computation to work.
And let me say one more thing. It seems that you are performing a genetic algorithm optimization, since you call your objective function a "fitness function." I believe you will have faster, more reliable results by using patternsearch instead of ga. The only difference in syntax is ga does not require a starting point, but patternsearch does. If you have finite lower and upper bound vectors lb and ub, you can get random initial points for patternsearch as follows:
x0 = lb + rand(size(lb)).*(ub - lb);
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
2 Comments
Jason Ross
on 31 Aug 2012
The plot thickens :)
I assume that there is a license manager of some sort running that the software communicates with, and that the license manager has a log file of the licenses being checked in and checked out. Can you get access to that log and see if it shows any errors about license counts being exceeded, etc?
See Also
Categories
Find more on Surrogate Optimization in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!