Quantcast

Documentation Center

  • Trial Software
  • Product Updates

Program Independent Jobs for a Supported Scheduler

Create and Run Jobs

This section details the steps of a typical programming session with Parallel Computing Toolbox™ software using a supported job scheduler on a cluster. Supported schedulers include the MATLAB job scheduler (MJS), Platform LSF® (Load Sharing Facility), Microsoft® Windows HPC Server (including CCS), PBS Pro®, or a TORQUE scheduler.

This section assumes you have anMJS, LSF®, PBS Pro, TORQUE, or Windows HPC Server (including CCS and HPC Server 2008) scheduler installed and running on your network. For more information about LSF, see http://www.platform.com/Products/. For more information about Windows HPC Server, see http://www.microsoft.com/hpc. With all of these cluster types, the basic job programming sequence is the same:

Note that the objects that the client session uses to interact with the MJS are only references to data that is actually contained in the MJS, not in the client session. After jobs and tasks are created, you can close your client session and restart it, and your job is still stored in the MJS. You can find existing jobs using the findJob function or the Jobs property of the MJS cluster object.

Define and Select a Profile

A cluster profile identifies the type of cluster to use and its specific properties. In a profile, you define how many workers a job can access, where the job data is stored, where MATLAB is accessed and many other cluster properties. The exact properties are determined by the type of cluster.

The step in this section all assume the profile with the name MyProfile identifies the cluster you want to use, with all necessary property settings. With the proper use of a profile, the rest of the programming is the same, regardless of cluster type. After you define or import your profile, you can set it as the default profile in the Profile Manager GUI, or with the command:

parallel.defaultClusterProfile('MyProfile')

A few notes regarding different cluster types and their properties:

    Notes   In a shared file system, all nodes require access to the folder specified in the cluster object's JobStorageLocation property.

    Because Windows HPC Server requires a shared file system, all nodes require access to the folder specified in the cluster object's JobStorageLocation property.

    In a shared file system, MATLAB® clients on many computers can access the same job data on the network. Properties of a particular job or task should be set from only one client computer at a time.

    When you use an LSF scheduler in a nonshared file system, the scheduler might report that a job is in the finished state even though the LSF scheduler might not yet have completed transferring the job's files.

Find a Cluster

You use the parcluster function to identify a cluster and to create an object representing the cluster in your local MATLAB session.

To find a specific cluster, user the cluster profile to match the properties of the cluster you want to use. In this example, MyProfile is the name of the profile that defines the specific cluster.

c = parcluster('MyProfile');
 MJS Cluster

   Properties
                            Name: my_mjs
                         Profile: MyProfile
                        Modified: false
                            Host: node345
                        Username: mylogin

                      NumWorkers: 1
                  NumBusyWorkers: 0
                  NumIdleWorkers: 1

              JobStorageLocation: Database on node345
               ClusterMatlabRoot: C:\apps\matlab
                 OperatingSystem: windows
                AllHostAddresses: 0:0:0:0
                   SecurityLevel: 0 (No security)
          HasSecureCommunication: false

   Associated Jobs

                  Number Pending: 0
                   Number Queued: 0
                  Number Running: 0
                 Number Finished: 0

Create a Job

You create a job with the createJob function. Although this command executes in the client session, it actually creates the job on the cluster, c, and creates a job object, job1, in the client session.

job1 = createJob(c)
 Job

    Properties:
                   ID: 1
                 Type: Independent
             Username: mylogin
                State: pending
           SubmitTime: 
            StartTime: 
     Running Duration: 0 days 0h 0m 0s

      AutoAttachFiles: true
  Auto Attached Files: List files
        AttachedFiles: {}
      AdditionalPaths: {}

    Associated Tasks:

       Number Pending: 0
       Number Running: 0
      Number Finished: 0
    Task ID of Errors: []

Note that the job's State property is pending. This means the job has not been queued for running yet, so you can now add tasks to it.

The cluster's display now includes one pending job, as shown in this partial listing:

c
Associated Jobs

                  Number Pending: 1
                   Number Queued: 0
                  Number Running: 0
                 Number Finished: 0

You can transfer files to the worker by using the AttachedFiles property of the job object. For details, see Share Code with the Workers.

Create Tasks

After you have created your job, you can create tasks for the job using the createTask function. Tasks define the functions to be evaluated by the workers during the running of the job. Often, the tasks of a job are all identical. In this example, each task will generate a 3-by-3 matrix of random numbers.

createTask(job1, @rand, 1, {3,3});
createTask(job1, @rand, 1, {3,3});
createTask(job1, @rand, 1, {3,3});
createTask(job1, @rand, 1, {3,3});
createTask(job1, @rand, 1, {3,3});

The Tasks property of job1 is now a 5-by-1 matrix of task objects.

job1.Tasks
         ID       State    FinishTime  Function  Error
 -----------------------------------------------------
    1     1     pending                   @rand       
    2     2     pending                   @rand       
    3     3     pending                   @rand       
    4     4     pending                   @rand       
    5     5     pending                   @rand       

Alternatively, you can create the five tasks with one call to createTask by providing a cell array of five cell arrays defining the input arguments to each task.

T = createTask(job1, @rand, 1, {{3,3} {3,3} {3,3} {3,3} {3,3}});

In this case, T is a 5-by-1 matrix of task objects.

Submit a Job to the Job Queue

To run your job and have its tasks evaluated, you submit the job to the job queue with the submit function.

submit(job1)

The job manager distributes the tasks of job1 to its registered workers for evaluation.

Each worker performs the following steps for task evaluation:

  1. Receive AttachedFiles and AdditionalPaths from the job. Place files and modify the path accordingly.

  2. Run the jobStartup function the first time evaluating a task for this job. You can specify this function in AttachedFiles or AdditionalPaths. When using an MJS, ff the same worker evaluates subsequent tasks for this job, jobStartup does not run between tasks.

  3. Run the taskStartup function. You can specify this function in AttachedFiles or AdditionalPaths. This runs before every task evaluation that the worker performs, so it could occur multiple times on a worker for each job.

  4. If the worker is part of forming a new parallel pool, run the poolStartup function. (This occurs when executing parpool or when running other types of jobs that form and use a parallel pool, such as batch.)

  5. Receive the task function and arguments for evaluation.

  6. Evaluate the task function, placing the result in the task's OutputArguments property. Any error information goes in the task's Error property.

  7. Run the taskFinish function.

Retrieve Job Results

The results of each task's evaluation are stored in that task object's OutputArguments property as a cell array. Use the function fetchOutputs to retrieve the results from all the tasks in the job.

wait(job1)
results = fetchOutputs(job1);

Display the results from each task.

results{1:5}
    0.9501    0.4860    0.4565
    0.2311    0.8913    0.0185
    0.6068    0.7621    0.8214

    0.4447    0.9218    0.4057
    0.6154    0.7382    0.9355
    0.7919    0.1763    0.9169

    0.4103    0.3529    0.1389
    0.8936    0.8132    0.2028
    0.0579    0.0099    0.1987

    0.6038    0.0153    0.9318
    0.2722    0.7468    0.4660
    0.1988    0.4451    0.4186

    0.8462    0.6721    0.6813
    0.5252    0.8381    0.3795
    0.2026    0.0196    0.8318

Manage Objects in the Scheduler

Because all the data of jobs and tasks resides in the cluster job storage location, these objects continue to exist even if the client session that created them has ended. The following sections describe how to access these objects and how to permanently remove them:

What Happens When the Client Session Ends

When you close the client session of Parallel Computing Toolbox software, all of the objects in the workspace are cleared. However, the objects in MATLAB Distributed Computing Server™ software or other cluster resources remain in place. When the client session ends, only the local reference objects are lost, not the actual job and task data in the cluster.

Therefore, if you have submitted your job to the cluster job queue for execution, you can quit your client session of MATLAB, and the job will be executed by the cluster. You can retrieve the job results later in another client session.

Recover Objects

A client session of Parallel Computing Toolbox software can access any of the objects in MATLAB Distributed Computing Server software, whether the current client session or another client session created these objects.

You create cluster objects in the client session by using the parcluster function.

c = parcluster('MyProfile');

When you have access to the cluster by the object c, you can create objects that reference all those job contained in that cluster. The jobs are accessible in cluster object's Jobs property, which is an array of job objects:

all_jobs = c.Jobs

You can index through the array all_jobs to locate a specific job.

Alternatively, you can use the findJob function to search in a cluster for any jobs or a particular job identified by any of its properties, such as its State.

all_jobs = findJob(c);
finished_jobs = findJob(c,'State','finished')

This command returns an array of job objects that reference all finished jobs on the cluster c.

Reset Callback Properties (MJS Only)

When restarting a client session, you lose the settings of any callback properties (for example, the FinishedFcn property) on jobs or tasks. These properties are commonly used to get notifications in the client session of state changes in their objects. When you create objects in a new client session that reference existing jobs or tasks, you must reset these callback properties if you intend to use them.

Remove Objects Permanently

Jobs in the cluster continue to exist even after they are finished, and after the MJS is stopped and restarted. The ways to permanently remove jobs from the cluster are explained in the following sections:

Delete Selected Objects.  From the command line in the MATLAB client session, you can call the delete function for any job or task object. If you delete a job, you also remove all tasks contained in that job.

For example, find and delete all finished jobs in your cluster that belong to the user joep.

c = parcluster('MyProfile')
finished_jobs = findJob(c,'State','finished','Username','joep')
delete(finished_jobs)
clear finished_jobs

The delete function permanently removes these jobs from the cluster. The clear function removes the object references from the local MATLAB workspace.

Start an MJS from a Clean State.  When an MJS starts, by default it starts so that it resumes its former session with all jobs intact. Alternatively, an MJS can start from a clean state with all its former history deleted. Starting from a clean state permanently removes all job and task data from the MJS of the specified name on a particular host.

As a network administration feature, the -clean flag of the startjobmanager script is described in Start in a Clean State in the MATLAB Distributed Computing Server System Administrator's Guide.

Was this topic helpful?