Learn more about MATLAB Distributed Computing Server

Using a Different MPI Build on UNIX Operating Systems

On this page…
Building MPI Using Your MPI Build

Building MPI

To use an MPI build that differs from the one provided with Parallel Computing Toolbox, this stage outlines the steps for creating an MPI build. If you already have an alternative MPI build, proceed to Using Your MPI Build.

1. Unpack the MPI sources into the target file system on your machine. For example, suppose you have downloaded mpich2-distro.tgz and want to unpack it into /opt for building:

   # cd /opt
   # mkdir mpich2 && cd mpich2
   # tar zxvf path/to/mpich2-distro.tgz
   # cd mpich2-1.0.8

2. Build your MPI using the enable-sharedlibs option (this is vital, as you must build a shared library MPI, binary compatible with MPICH2-1.0.8 for R2009b and later). For example, the following commands build an MPI with the nemesis channel device and the gforker launcher.

   # ./configure -prefix=/opt/mpich2/mpich2-1.0.8 \
    --enable-sharedlibs=gcc \
    --with-device=ch3:nemesis \
    --with-pm=gforker 2>&1 | tee log
   # make 2>&1 | tee -a log
   # make install 2>&1 | tee -a log

Back to Top

Using Your MPI Build

When your MPI build is ready, this stage highlights the steps to use it. To get the Parallel Computing Toolbox mpiexec scheduler working with a different MPI build, follow these steps. Most of these steps are also needed if you want to use a different MPI build with third party-schedulers (LSF, generic).

1. Test your build by running the mpiexec executable. The build should be ready to test if its bin/mpiexec and lib/libmpich.so are available in the MPI installation location.

   Following the example in Building MPI, /opt/mpich2/mpich2-1.0.8/bin/mpiexec and /opt/mpich2/mpich2-1.0.8/lib/libmpich.so are ready to use, so you can test the build with:

   $ /opt/mpich2/mpich2-1.0.8/bin/mpiexec -n 4 hostname

2. Create an mpiLibConf function to direct Parellel Computing Toolbox to use your new MPI. Write your mpiLibConf.m to return the appropriate information for your build. For example:

   function [primary, extras] = mpiLibConf
   primary = '/opt/mpich2/mpich2-1.0.8/lib/libmpich.so';
   extras  = {};

   The primary path must be valid on the cluster; and your mpiLibConf.m file must be higher on the cluster workers' path than matlabroot/toolbox/distcomp/mpi. (Sending mpiLibConf.m as a file dependency for this purpose does not work. You can get the mpiLibConf.m function on the worker path by either moving the file into a directory on the path, or by having the scheduler use cd in its command so that it starts the MATLAB worker from within the directory that contains the function.)

3. Determine necessary daemons and command-line options.

   • Determine all necessary daemons (often something like mpdboot or smpd). The gforker build example in this section uses an MPI that needs no services or daemons running on the cluster, but it can use only the local machine.

   • Determine the correct command-line options to pass to mpiexec.

4. Use one of the following options to set up your scheduler to use your new MPI build:

   • For the simplest case of the mpiexec scheduler, set up a configuration to use the mpiexec executable from your new MPI build. It is crucial that you use matching mpiexec, MPI library, and any daemons (if any), together. Set the configuration's MpiexecFileName property to /opt/mpich2/mpich2-1.0.8/bin/mpiexec.

   • If you are using a generic scheduler or LSF, modify your parallel wrapper script to pick up the correct mpiexec. Additionally, there may be a stage in the wrapper script where the MPI daemons are launched.

     The parallel submission wrapper script must:

     • Determine which nodes are allocated by the scheduler.

     • Start required daemon processes. For example, for the MPD process manager this means calling "mpdboot -f <nodefile>".

     • Define which mpiexec executable to use for starting workers.

     • Stop the daemon processes. For example, for the MPD process manager this means calling "mpdallexit".

     For examples of parallel wrapper scripts, see matlabroot/toolbox/distcomp/examples/integration/; specifically for an example of Sun Grid Engine, look in the folder sge for sgeParallelWrapper.sh. Adopt and modify the appropriate script for your particular cluster usage.

Back to Top

Installing and Configuring

Shutting Down a Job Manager Configuration

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS