Clear Filters
Clear Filters

How is the work distributed in parfor loop?

1 view (last 30 days)
In a scenario with a parfor loop consisting of 10000 iterations distributed over 10 workers, is each worker initially assigned 10000/10=1000 tasks at the beginning of the parfor loop, or does each worker receive one task initially and subsequently get assigned new tasks upon completion?
Specifically, I am encountering a situation where approximately 500 iterations of a parfor loop (of total 10000 iterations) are computationally heavy while the remaining ~9500 iterations are computationally lighter. When running this parfor loop, I notice that the execution is initially extremely fast, with around 90% completion within 5 minute. However, the remaining 10% takes significantly longer, up to 20 hours. My suspicion is that the computationally heavy iterations are being assigned to a single worker, which handles them sequentially.

Accepted Answer

Edric Ellis
Edric Ellis on 16 Feb 2024
If you know in advance which iterations are the time-consuming ones, then you could consider running those together as a separate parfor loop, and the default iteration partitioning described by @Walter Roberson will probably do a decent job of keeping all workers busy.
If you do not know in advance, then there are two strategies that you could use, and maybe you could use both together:
  1. Shuffle the order of the iterations using randperm or similar to try and end up with the long-running iterations in different sub-ranges.
  2. Use parforOptions to force small sub-ranges.
Luqman Saleem
Luqman Saleem on 17 Feb 2024
Thank you. I have a general idea of which iterations are computationally intensive. My current parfor loop structure is as follows:
parfor ix = 1:10000
% code
Specifically, I have identified the iterations from ix = 2000 to 2500 and from ix = 7000 to 7500 as computationally heavy. Could you please provide a sample code illustrating how I can instruct the parfor loop to distribute these iterations among multiple workers?
Walter Roberson
Walter Roberson on 17 Feb 2024
For example, set the RangePartitionMethod to [50*ones(1,(1950-0)/50), 20*ones(1,(2500-2000)/50), 50*ones(1,(7000-2500)/50)), 20*ones(1,(7500-7000)/20), 50*ones(1,(10000-7500)/50)]
or something like that.

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 15 Feb 2024
You can control how parfor divides up the iterations by using parforOptions() RangePartitionMethod
The default is "auto", which divides most iterations up into chunks, then divides most of the remaining iterations into smaller chunks, then hands out the remaining iterations one at a time.


Find more on Parallel for-Loops (parfor) in Help Center and File Exchange





Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!