Why does my parallel processing throttle down and hang for the last portion of jobs?

I am processing a parallel job that runs over something like 234 images. While the first 210 or so process in reasonable amount of time (~3 hours). The last 24 images took around 13 hours. Each image after the initial 210 would finishing processing every 40 minutes to over an hour. Does the parallel computing need a number of jobs divisible by the number of workers?

Here is a simplified version of my code:

p = gcp('nocreate'); % If no pool, do not create new one.
if isempty(p)
  parpool(28);
else
  delete(p);
  parpool(28);
end

% Process the selected images
for k = 1:length(patches)
    curr_patch = patches(k);
    file_path = fullfile(test_data_path, file_group, 'ProductGeneration', sprintf('Patch%03d',curr_patch));
    im_list = dir(fullfile(file_path, '*_DCP.mat'));

    % Create a folder for each patch if it doesn't exist
    if(~exist(['.\Results\' sprintf('Patch%03d',curr_patch)],'dir'))
        mkdir(['.\Results\' sprintf('Patch%03d',curr_patch)])
    end

    parfor m = 1:length(im_list)
        % Read in the image
        im_name = im_list(m).name;
        S = load(fullfile(file_path, im_name));
        cimg = S.cimg;

        result = process_image(cimg);

        % Make RGB image that puts results over the original image
        rgb = make_rgb_image(cimg, result);

        % Write RGB image to .tif file and save original image and mask in
        % .mat
        [~, fname, ~] = fileparts(im_name);
        res_path = ['.\Results\' sprintf('Patch%03d',curr_patch) '\'];
        imwrite(rgb,[res_path fname '_mask.tif'],'tif');
        % Saves .mat file
        parsave([res_path im_name], cimg, result);
     end
end

Thanks for your help in advance.

-Jon