Threads based parfor throws feval error with seemingly random behaviour

Hi!

I've got a question that hopefully someone can help me with. I'm experiencing some strange behaviour from a parfor loop in my code. To give a very simplified version of the code segment, it looks something like this:

my_cell_array = cell(N,1);
parpool('Thread')
for i = 1:N
    my_cell_array{i} = zeros(size_output(i),1); % Preallocating the cell array, although whether I do or do not preallocate makes no difference in terms of the error behaviour
end
parfor i = 1:N
    my_cell_array{i} = some_func(i,a_bunch_of_other_stuff);  % Expensive function
end

If I run this loop as is it usually works fine. However, even about 1 in 1000 times, I get the following error:

identifier: 'MATLAB:feval:argMustBeStringOrHandle'
message: "Function to evaluate must be represented as a string scalar, character vector, or function_handle object."

This occurs at a random frequency, even if a_bunch_of_other_stuff remains constant, i.e.,

my_cell_array = cell(N,1);
parpool('Thread')
for n = 1:10000
    rng(1)  
    for i = 1:N
        my_cell_array{i} = zeros(size_output(i),1); % Preallocating the cell array, although whether I do or do not preallocate makes no difference in terms of the error behaviour
    end
    parfor i = 1:N
        my_cell_array{i} = some_func(i,a_few_other_inputs); % Expensive function
    end
end

This always errors eventually, sometimes as early as n == 20, sometimes as late as n == 2000, but usually around n == 200.

If I run this with a parpool with the 'Processess' profile, it never errors. However, in my code I have a different parfor loop in which using the "Threads" profile is much preferred as there is a lot of data transfer costs when I use a "Processes" parpool.

The function is rather complex, so I apologize for not giving more details about it, however the error definitely does not occur within the function. i.e.,

my_cell_array = cell(N,1);
parpool('Thread')
for n = 1:10000
    parfor i = 1:N
        for i = 1:N
            my_cell_array{i} = zeros(size_output(i),1); % Preallocating the cell array, although whether I do or do not preallocate makes no difference in terms of the error behaviour
        end
        try
        my_cell_array{i} = some_func(i,a_few_other_inputs); % Expensive function
        catch e
            error("THE ERROR OCCURED INSIDE THE FUNCTION"); % This never occurs
        end    
    end
end

With parfors it can be rather difficult to debug, so I'm wondering if anyone could explain:

1) What is it about running the "Threads" parpool that allows for the possibility of an error

2) Why it is is possible for the error to occur only occasionally (and with random behaviour)

3) Is there something within my funciton that I should be looking for that could be causing this behaviour?

Thanks in advance for any help that you can provide!

[dpb]

Certainly don't think anybody can address any of the above (certainly not 1 and 3, anyway) without the code to look at. 2) is perhaps related to some sort of timing/resource allocation so does show variability.

My guess is the only way to get any meaningful support here would be to build a minimal working example that exhibits the behavior and submit it to TMW for support.

Can you illustrate the behavior with any function other than this particular one? If so, that would be pretty good evidence for something inside the parallel dispatcher; if not, then that would be indicative that while the error may not actually be inside your function, something the function itself is doing is responsible for creating the error condition.

The only way for anybody here to be able to have any chance at all would be the same thing -- build a minimal working example that illustrates the problem and attach it including any and all data needed.

[Charles]

Okay, I thought that might be the case. I quickly tried to build a mwe but it didn't exhibit the same behaviour so at the moment I will just forgo the parfor for this section of my code. If I find the time I will spend some more time trying to build a mwe and will ask again once I get it (not) working.

[dpb]

"... tried to build a mwe but it didn't exhibit the same behaviour"

AHA! That's informative, methinks and implies the content and/or structure of the function is, in fact, the root cause even though the function itself doesn't error.

Perhaps it would be possible to remove pieces of the function in something like a binary sectioning process to see if could isolate within the function what might be the area containing the symptom-causing code.

[Charles]

Oh that's a great idea. Yeah I'll just progressively remove piece of the function until it stops erroring. It seems obvious now that I type it out!

[Edric Ellis]

@Charles it may be that what you're seeing here is a bug that was fixed in R2023b. Are you able to upgrade to that release and try again?