FileDependencies - Directories and files that worker can access

Description

FileDependencies contains a list of directories and files that the worker will need to access for evaluating a job's tasks.

The value of the property is defined by the client session. You set the value for the property as a cell array of strings. Each string is an absolute or relative pathname to a folder or file. The toolbox makes a zip file of all the files and directories referenced in the property. (Note: If the files or directories change while they are being zipped, or if any of the directories are empty, a failure or error can result.)

The first time a worker evaluates a task for a particular job, the scheduler passes to the worker the zip file of the files and directories in the FileDependencies property. On the worker machine, the file is unzipped, and a folder structure is created that is exactly the same as that accessed on the client machine where the property was set. Those entries listed in the property value are added to the top of the path in the MATLAB worker session. (The subdirectories of the entries are not added to the path, even though they are included in the folder structure.) To find out where the unzipping occurs on the worker machine, use the function getFileDependencyDir in code that runs on the worker. See Example 2, below.

When the worker runs subsequent tasks for the same job, it uses the folder structure already set up by the job's FileDependencies property for the first task it ran for that job.

When you specify FileDependencies at the time of creating a job, the settings are combined with those specified in the applicable configuration, if any. (Setting FileDependencies on a job object after it is created does not combine the new setting with the configuration settings, but overwrites existing settings for that job.)

The transfer and unzipping of FileDependencies occurs for each worker running a task for that particular job on a machine, regardless of how many workers run on that machine. Normally, the file dependencies are deleted from the worker machine when the job is completed, or when the next job begins.

Characteristics

Usage	Job object
Read-only	After job is submitted
Data type	Cell array of strings

Values

The value of FileDependencies is empty by default. If a pathname that does not exist is specified for the property value, an error is generated.

Remarks

There is a default limitation on the size of data transfers via the AttachedFiles property. For more information on this limit, see Object Data Size Limitations. For alternative means of making data available to workers, see Share Code.

Examples

Example 1

Make available to a job's workers the contents of the directories fd1 and fd2, and the file fdfile1.m.

set(job1,'FileDependencies',{'fd1' 'fd2' 'fdfile1.m'})
get(job1,'FileDependencies')
ans = 
    'fd1'
    'fd2'
    'fdfile1.m'

Example 2

Suppose in your client MATLAB session you have the following folders on your MATLAB path:

dirA
dirA\subdir1
dirA\subdir2
dirB

Transfer the content of these folders to the worker machines, and add all these folders to the paths of the worker MATLAB sessions. On the client, execute the following code:

j = createJob(FileDependencies, {'dirA', 'dirB'})
% This includes the subfolders of dirA.

In the task function that executes on the workers, include the following code:

%First find where FileDependencies are unzipped:
  DepDir = getFileDependencyDir
%The top folders are already on the path, so add subfolders:
  addpath(fullfile(DepDir,'dirA','subdir1'),...
          fullfile(DepDir,'dirA','subdir2'))

See Also

Functions

getFileDependencyDir, jobStartup, taskFinish, taskStartup

Properties

PathDependencies

© 1984-2012- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS