- Parallel Computing Toolbox ( https://www.mathworks.com/products/parallel-computing.html)
- MATLAB Distributed Computing Server ( https://www.mathworks.com/products/distriben.html)
How to run MATLAB on a Sun Grid Engine cluster to use parallel optimization
15 views (last 30 days)
Show older comments
I am trying to use the Global Optimization Toolbox in MATLAB to minimize a non-smooth function. The function computation is quite expensive and hence it makes sense to find local minimas by starting from different points in parallel.
I am trying to run my matlab script on a cluster which uses the Sun Grid Engine for scheduling jobs. However, it seems that I am capped by the number of cores in each node to run parallel optimizations. Is there any way to share cores across nodes to run my job in parallel with several cores?
0 Comments
Answers (1)
Kojiro Saito
on 24 Feb 2017
You need 2 more products to run parallel job in a cluster.
Parallel Computing Toolbox enables you to run parallel jobs in a local machine. In adddition to it with MATLAB Distributed Computing Server, you can run parallel jobs on cluster nodes.
About, Grid Engine can be integrated with these 2 products with the following support package. https://www.mathworks.com/matlabcentral/fileexchange/52816-parallel-computing-toolbox-integration-for-matlab-distributed-computing-server-with-grid-engine
3 Comments
Kojiro Saito
on 2 Mar 2017
You can find whether you have a MATLAB Distributed Computing Server license in the license file (license.lic or license.dat) if "INCREMENT MATLAB_Distrib_Comp_Engine" line is written in the file.
Also, You can find whether MATLAB Distributed Computing Server is installed on the server if $MATLAB_INSTALL/toolbox/distcomp/bin/mdce (if UNIX) or $MATLAB_INSTALL\toolbox\distcomp\bin\mdce.bat (if Windows) is existing.
Without MATLAB Distribued Computing Server, I have no good idea. Maybe you can implement by using MPI across nodes...
Walter Roberson
on 2 Mar 2017
It is possible to add MPI calls to transfer data, either through loadlibrary() or through a mex call. Or you could use tcp() or udp() to communicate. Or if you have a common file system, you could use memmapfile to communicate.
If you are transferring MATLAB data structures, you will have to find some way to serialize them. The official serialization routines are not exposed, but you can find a decent "fast serialize" contribution in the File Exchange.
See Also
Categories
Find more on MATLAB Parallel Server in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!