RemoteClusterAccess

Connect to schedulers when client utilities are not available locally

Constructor

parallel.cluster.RemoteClusterAccess

r = parallel.cluster.RemoteClusterAccess(username)

r = parallel.cluster.RemoteClusterAccess(username,P1,V1,...,Pn,Vn)

Description

parallel.cluster.RemoteClusterAccess allows you to establish a connection and run commands on a remote host. This class is intended for use with the generic scheduler interface when using remote submission of jobs or on nonshared file systems.

r = parallel.cluster.RemoteClusterAccess(username) uses the supplied username when connecting to the remote host, and returns a RemoteClusterAccess object r. You will be prompted for a password when establishing the connection.

r = parallel.cluster.RemoteClusterAccess(username,P1,V1,...,Pn,Vn) allows additional parameter-value pairs that modify the behavior of the connection. The accepted parameters are:

  • 'IdentityFilename' — A character vector containing the full path to the identity file to use when connecting to a remote host. If 'IdentityFilename' is not specified, you are prompted for a password when establishing the connection.

  • 'IdentityFileHasPassphrase' — A logical indicating whether or not the identity file requires a passphrase. If true, you are prompted for a password when establishing a connection. If an identity file is not supplied, this property is ignored. This value is false by default.

For more information and detailed examples, see the integration scripts provided in matlabroot/toolbox/distcomp/examples/integration. For example, the scripts for PBS in a nonshared file system are in

matlabroot/toolbox/distcomp/examples/integration/pbs/nonshared

Methods

Method NameDescription
connect

connect(r,clusterHost) establishes a connection to the specified host using the user credential options supplied in the constructor. File mirroring is not supported.

connect(r,clusterHost,remoteDataLocation) establishes a connection to the specified host using the user credential options supplied in the constructor. remoteDataLocation identifies a folder on the clusterHost that is used for file mirroring. The user credentials supplied in the constructor must have write access to this folder.

disconnectdisconnect(r) disconnects the existing remote connection. The connect method must have already been called.
doLastMirrorForJob

doLastMirrorForJob(r,job) performs a final copy of changed files from the remote DataLocation to the local DataLocation for the supplied job. Any running mirrors for the job also stop and the job files are removed from the remote DataLocation. The startMirrorForJob or resumeMirrorForJob method must have already been called.

getRemoteJobLocation

getRemoteJobLocation(r,jobID,remoteOS) returns the full path to the remote job location for the supplied jobID. Valid values for remoteOS are 'pc' and 'unix'.

isJobUsingConnection

isJobUsingConnection(r,jobID) returns true if the job is currently being mirrored.

resumeMirrorForJob

resumeMirrorForJob(r,job) resumes the mirroring of files from the remote DataLocation to the local DataLocation for the supplied job. This is similar to the startMirrorForJob method, but does not first copy the files from the local DataLocation to the remote DataLocation. The connect method must have already been called. This is useful if the original client MATLAB session has ended, and you are accessing the same files from a new client session.

runCommand

[status,result] = runCommand(r,command) runs the supplied command on the remote host and returns the resulting status and standard output. The connect method must have already been called.

startMirrorForJob

startMirrorForJob(r,job) copies all the job files from the local DataLocation to the remote DataLocation, and starts mirroring files so that any changes to the files in the remote DataLocation are copied back to the local DataLocation. The connect method must have already been called.

stopMirrorForJob

stopMirrorForJob(r,job) immediately stops the mirroring of files from the remote DataLocation to the local DataLocation for the specified job. The startMirrorForJob or resumeMirrorForJob method must have already been called. This cancels the running mirror and removes the files for the job from the remote location. This is similar to doLastMirrorForJob, except that stopMirrorForJob makes no attempt to ensure that the local job files are up to date. For normal mirror stoppage, use doLastMirrorForJob.

getConnectedAccess

getConnectedAccess(host,username) returns a RemoteClusterAccess object that is connected to the supplied host. This function may return a previously constructed RemoteClusterAccess object if one exists.

getConnectedAccess(...,P1,V1,...Pn,Vn) passes the additional parameters to the RemoteClusterAccess constructor.

getConnectedAccessWithMirror

getConnectedAccessWithMirror(host,location,username) returns a RemoteClusterAccess object that is connected to the supplied host, using the location as the mirror location. This function may return a previously constructed RemoteClusterAccess object if one exists.

getConnectedAccessWithMirror(...,P1,V1,...Pn,Vn) passes the additional parameters to the RemoteClusterAccess constructor.

Properties

A RemoteClusterAccess object has the following read-only properties. Their values are set when you construct the object or call its connect method.

Property NameDescription
HostnameName of the remote host to access.
IdentityFileHasPassphrase Indicates if the identity file requires a passphrase.
IdentityFilenameFull path to the identity file used when connecting to the remote host.
IsConnectedIndicates if there is an active connection to the remote host.
IsFileMirrorSupportedIndicates if file mirroring is supported for this connection. This is false if no remote DataLocation is supplied to the connect() method.
JobStorageLocationLocation on the remote host for files that are being mirrored.
UseIdentityFileIndicates if an identity file should be used when connecting to the remote host.
UsernameUser name for connecting to the remote host.

Examples

Mirror files from the remote data location. Assume the object job represents a job on your generic scheduler.

remoteConnection = parallel.cluster.RemoteClusterAccess('testname');
connect(remoteConnection,'headnode1','/tmp/filemirror');
startMirrorForJob(remoteConnection,job);
submit(job)
% Wait for the job to finish
wait(job);

% Ensure that all the local files are up to date, and remove the 
% remote files
doLastMirrorForJob(remoteConnection,job);

% Get the output arguments for the job
results = fetchOutputs(job)

For more detailed examples, see the integration scripts provided in matlabroot/toolbox/distcomp/examples/integration. For example, the scripts for PBS in a nonshared file system are in

matlabroot/toolbox/distcomp/examples/integration/pbs/nonshared

Introduced in R2011a