If you have a for loop which can be written as:
for a = 1:size(input, 2)
output(:,a) = func(input(:,a), global_data);
where input is a numeric array and output is a numeric or cell array, then batch_job_distrib() can parallelize the work across multiple worker MATLAB instances on multiple (unlimited) networked worker PCs as follows:
output = batch_job_distrib(func, input, workers, global_data);
This is a replacement for parfor in this use case, if you don't have the Parallel Computing Toolbox and/or Distributed Computing Server.
There is also an asynchronous mode, which returns immediately, passing back a handle to a function which can load the output data later:
output = batch_job_distrib(..., '-async'); % Start an asynchronous computation
... % Do other stuff here
output = output(); % Get the results here
The function can always spread the work across multiple MATLABs on the local PC, but the requirements for it to run on OTHER PCs are that:
- There is an ssh executable on the system path of the local PC.
- All worker PCs can be ssh'd into non-interactively (i.e. without manually entering a password).
- MATLAB is on the system path of all worker PCs.
- Every worker has a valid license for MATLAB and all required toolboxes for the user ssh'ing in.
- The current directory can be accessed on all worker PCs via the SAME path.
- The user can write to the current directory from all worker PCs.
- All the required functions are on the MATLAB paths of every worker PC.
- The networked filesystem supports file locks (not crucial, but safer).
- All the worker PCs honour the networked filesystem file locks (again, not crucial, but safer).
Note that the workers need not all run the same operating system, but they must all have working versions of the required functions, including where these are platform-dependent, e.g. mex files.
This function is in beta testing. If you have any bug reports or fixes, please send them to me via the 'Contact Author' link on my author page.