Get from Ico-github-logo

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Highlights from
Batch Job

  • batch_job(varargin)
    BATCH_JOB Run a batch job across several instances of MATLAB on the same PC
  • batch_job_collect(s, vara...
    BATCH_JOB_COLLECT Output data computed by a batch job
  • batch_job_distrib(varargin)
    BATCH_JOB_DISTRIB Distribute a MATLAB for loop across several PCs
  • batch_job_submit(job_dir,...
    BATCH_JOB_SUBMIT Submit a batch job to workers
  • batch_job_test
    Test two methods work and give the same result as a normal for loop
  • batch_job_worker(job)
    BATCH_JOB_WORKER Create a batch_job worker
  • chunk_name(work_dir, ind)
  • construct_function(s)
    Get the global data
  • do_chunk(func, mi, s, a)
    Get the chunk filename
  • do_job(job, kill)
  • getComputerName()
    GETCOMPUTERNAME returns the name of the computer (hostname)
  • get_chunk_indices(a, s)
  • get_file_lock(fname, force)
    GET_FILE_LOCK Grab a file lock for exclusive access to a file
  • kill_process(pid)
  • kill_signal(s)
    Signalled if the params file is deleted
  • matrify(output)
    Convert the output to a matrix or array
  • num_bytes(A)
  • open_mmap(mmap)
  • preallocate_file(name, nb...
    Allocate the right amount of space
  • quiet_delete(varargin)
  • set_chunk_time(job)
    Do the job initializations
  • setup_job(job)
    CD to the correct directory
  • start_workers(s, workers)
    Start the workers
  • timestr(t)
    Time string function
  • tmpname()
  • write_bin(A, fname)
  • write_launch_script(funCa...
    Save the command script
  • View all files
5.0 | 1 rating Rate this file 9 Downloads (last 30 days) File Size: 40 KB File ID: #44077 Version: 1.5
image thumbnail

Batch Job



24 Oct 2013 (Updated )

Toolbox to spread batch work over multiple MATLAB instances, even on other PCs

| Watch this File

File Information

This toolbox consists of several functions for spreading work over multiple MATLAB instances.
If you have a for loop which can be written something like:
   for a = 1:size(input, 2)
       output(:,a) = func(input(:,a), global_data);

then the functions in this toolbox can be used to split the for loop iterations across multiple MATLAB instances, including across multiple computing nodes/computers. This is a replacement for parfor in this use case, if you don't have the Parallel Computing Toolbox.

For the toolbox to work, its root directory needs to be on your MATLAB path at startup.

See the README at:
for more information.

These functions are in beta testing. If you have any bug reports or fixes, please raise a github issue.


Multicore Parallel Processing On Multiple Cores inspired this file.

Required Products MATLAB
MATLAB release MATLAB 8.1 (R2013a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (7)
18 Oct 2016 grega

grega (view profile)

batch_job_distrib works well on single PC. I'm trying to make it work in conjunction with another PC via batch_job_worker('shared_directory'). The message I'm getting is:

Could not copy batch script to host XXX
Error using start_workers (line 11)
'cat' is not recognized as an internal or external command, operable program or batch file.


Could not delete batch script on host XXX
Error using batch_job_distrib>cleanup_all (line 232)
'ssh' is not recognized as an internal or external command, operable program or batch file.

Please suggestions?

Comment only
08 Mar 2016 massa64

very nice work.
I had faced couple of issues in batch_job but I dont know if they were resolved.
1. matlab crashed when trying to map the memfile (at least that was what matlab was telling me), what was really going on was that with the '-automation' option matlab was trying to start an interactive session. I've resolved this with the '-nodisplay -nosplash -nodesktop', also suggested by Andreas. (around line 145 in batch_job)
2. I've workaround the variable input since in my case I can calculate maximum possible number of output and replaced the complement with 'NaNs' in the custom function.
If you can't calculate the max number of outcomes, than you can estimate with reasonable reserve, than try catch if overflows..
Hope this help a little

Comment only
23 Dec 2015 b

b (view profile)

Hello Oliver, I need a somewhat modified version of this script. The first instance of matlab runs - performs computations and then quits. THEN (after the first instance has shutdown) the second instance should start (automatically) - perform computations (with perhaps different set of variables) and quit. Once the 2nd instance has shutdown, THEN the 3rd instance starts and so on. How (and where) to change the batch_job file? Thanks.

Comment only
23 Oct 2014 David T_

First, thank you a lot for this contribution.

However, I'm facing variable size output data. Like Matthew I'm hoping on cell array support.
(Similar to CELLFUN(__, 'UniformOutput', false) )

Comment only
24 Jul 2014 Matthew

(This looks like a great piece of work! Thanks!)

It appears that this does not support non-numeric input and output, despite the markdown file stating that cell arrays can be used. I am ideally hoping to pass structs in and out (though cell arrays would also be sufficient) -- is this possible?

Comment only
16 Feb 2014 almog shalom

Very nice.
I would add to the command that invokes Matlab the option "-nosplash", and maybe also "-nodisplay".

25 Oct 2013 Andreas

Please add an example in the comment section.

Comment only
29 Oct 2013 1.1

Add example. Improve code.

30 Oct 2013 1.2

Add the logo

11 Dec 2013 1.3

Bug fix

18 Aug 2015 1.4

Move to github

23 Dec 2015 1.5

Update description

02 Jan 2016 1.5

Update help text

Contact us