Code covered by the BSD License  

Highlights from
Progress monitor (progress bar) that works with parfor

4.72727

4.7 | 11 ratings Rate this file 93 Downloads (last 30 days) File Size: 2.21 KB File ID: #32101

Progress monitor (progress bar) that works with parfor

by

 

06 Jul 2011 (Updated )

Displays the percentage of parfor iterations which have completed.

| Watch this File

File Information
Description

Monitoring the progress of a long-running parfor loop is not straightforward because the workers cannot communicate. This function gets around that limitation by storing the progress in a text file.

Example usage:

N = 100;
parfor_progress(N); % Initialize
parfor i=1:N
    pause(rand); % Replace with real code
    parfor_progress; % Count
end
parfor_progress(0); % Clean up

Acknowledgements

This file inspired Progress Bar For Matlab Loops (Incl. Parfor) and Timed Progress Bar.

Required Products Parallel Computing Toolbox
MATLAB release MATLAB 7.10 (R2010a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (15)
28 May 2014 James  
16 May 2014 Jeremy

Erik: Depends how slow each iteration of your loop is. If each iteration takes a while to run, this won't impact execution time. If each iteration is very fast, it could. You'll have to benchmark for more specifics, I've only used it on slow problems. If necessary, a quick fix to speed this up would be to change the path of parfor_progress.txt to be on a ramdisk.

15 May 2014 Erik

Nice work. Just wondering, what impact does this have on the execution speed of parfor?

14 Oct 2013 Hassan Naseri  
06 Oct 2013 Jesse Coombs

Would it be possible to get a version that instead prints the %complete to a txt file for use in monitoring jobs on remote/server pc's?

02 Jul 2013 Alessandro Corbetta

Excellent work!
Although it works badly on linux -nodesktop mode. In this case, the older (and less complete) bars remain showed in the console.
As a consequence, the screen is full of more or less complete version of the bar in few seconds.

10 Jun 2013 Antoine Liutkus  
02 Jun 2013 German

Thanks for the file. It works!
As Timo suggested, if there are several instances of MATLAB running (e.g., multi-core server), it won't work properly.
It can be easily solved by adding a randon sequence to the file name.

function percent = parfor_progress(N)
%PARFOR_PROGRESS Progress monitor (progress bar) that works with parfor.
% PARFOR_PROGRESS works by creating a file called parfor_progress.txt in
% your working directory, and then keeping track of the parfor loop's
% progress within that file. This workaround is necessary because parfor
% workers cannot communicate with one another so there is no simple way
% to know which iterations have finished and which haven't.
%
% PARFOR_PROGRESS(N) initializes the progress monitor for a set of N
% upcoming calculations.
%
% PARFOR_PROGRESS updates the progress inside your parfor loop and
% displays an updated progress bar.
%
% PARFOR_PROGRESS(0) deletes parfor_progress.txt and finalizes progress
% bar.
%
% To suppress output from any of these functions, just ask for a return
% variable from the function calls, like PERCENT = PARFOR_PROGRESS which
% returns the percentage of completion.
%
% Example:
%
% N = 100;
% parfor_progress(N);
% parfor i=1:N
% pause(rand); % Replace with real code
% parfor_progress;
% end
% parfor_progress(0);
%
% See also PARFOR.

% By Jeremy Scheff - jdscheff@gmail.com - http://www.jeremyscheff.com/
error(nargchk(0, 1, nargin, 'struct'));

persistent fname;

if nargin < 1
N = -1;
end

percent = 0;
w = 50; % Width of progress bar

if N > 0
rndseq = round(10^6*rand(1));
fname = strcat('parfor_progress_',num2str(rndseq),'.txt');

f = fopen(fname, 'w');
if f<0
error('Do you have write permissions for %s?', pwd);
end
fprintf(f, '%d\n', N); % Save N at the top of progress.txt
fclose(f);

if nargout == 0
disp([' 0%[>', repmat(' ', 1, w), ']']);
end
elseif N == 0
delete(fname);
percent = 100;

if nargout == 0
disp([repmat(char(8), 1, (w+9)), char(10), '100%[', repmat('=', 1, w+1), ']']);
end
else
if ~exist(fname, 'file')
error([fname,' not found. Run PARFOR_PROGRESS(N) before PARFOR_PROGRESS to initialize ',fname]);
end

f = fopen(fname, 'a');
fprintf(f, '1\n');
fclose(f);

f = fopen(fname, 'r');
progress = fscanf(f, '%d');
fclose(f);
percent = (length(progress)-1)/progress(1)*100;

if nargout == 0
perc = sprintf('%3.0f%%', percent); % 4 characters wide, percentage
disp([repmat(char(8), 1, (w+9)), char(10), perc, '[', repmat('=', 1, round(percent*w/100)), '>', repmat(' ', 1, w - round(percent*w/100)), ']']);
end
end

14 May 2013 Timo

This is a great function you've implemented, but there is one point you could advance.

Here is the problem:
If you execute any MATLAB script more than once in the same folder, and the script contains your function, they will end up in an error.
The point is, that the filename is always the same which is created by your function.
So you'll have more than one function which tries to work on the parfor_progress file.

Maybe you should choose an arbitrary filename-id attached to every parfor_progress file.

But anyway, great work!

10 Mar 2013 Michael  
08 Aug 2012 Donald Bucci

This is real wonderful, however I had some issues running this on a nodisplay ubuntu server terminal running matlab (ascii 8 for backspace doesn't quite produce the right output).

If I were you, I'd rewrite everything in here from DISP() functions into FPRINTF() functions. And instead of using char(8) and char(10) for your backspaces and carriage returns, use the breakout switches \b and \n respectively. I did that and it fixed any issues I've ever had with the progress bar being duplicated across multiple lines (as opposed to being overwritten when updated).

13 Jul 2012 Pantelis  
05 Mar 2012 Jeremy

Ashin,

I develop on Linux and I've never seen this problem. So it's likely that it is not a Linux problem in general, but it's specific to your particular setup. Maybe something to do with the character encoding on your system?

As a test, run this and see what it outputs:

disp(['aaa', char(8)])

The output should be just aa, because char(8) is a backspace. This is the technique I used in parfor_progress to delete the previous line, so maybe the backspace isn't working on your Linux machine for some reason?

Or does anyone know of a more universally portable way of doing this?

04 Mar 2012 Ashin Mukherjee

I'm trying to use this in a linux environment. The resulting progress bar looks something like this

0% [ ]
2% [= ]
4% [== ]
.
.
.

That is it creates a new-line every time which is slightly annoying. But works smoothly in windows. Was wondering whether there is an easy fix to this problem.

Thanks
Ashin

04 Mar 2012 Mike

Thanks - does exactly what it's supposed to, stops you scratching your head and wondering if it's nearly finished...

Updates
07 Jul 2011

Slightly simplified the function.

25 Oct 2011

It now displays a real (and pretty) progress bar.

Contact us