File operations in parfor loop lead to file operations error randomly
17 views (last 30 days)
Show older comments
Martin Schinnerl
on 20 Sep 2023
Commented: Martin Schinnerl
on 20 Sep 2023
I'm currently processing black & white images for a large number of experiments (datasets).
Therefore, a function called bwReadout is called for all datasets:
for ii = 1:nbr_datasets
[status(1,ii),output_id,input_ids] = bwReadout(dataset_name,...
input_subfolder_ids);
end
The relevant lines of bwReadout are:
function [status,output_ids,input_ids] = bwReadout(dataset_name,...
input_subfolder_ids)
% ...
parfor ii = 2:nbr_png
% . open *.png file:
fileID = fopen(fullfile(png_files(ii).folder,...
png_files(ii).name),'r');
% . skip 20 byte header and move to specified position i.e.
% 20 bytes in file starting at beginning of file - 'bof':
fseek(fileID,20,'bof');
% . read *.png file:
file_data = fread(fileID,'uint8=>uint8');
% . close *.png file:
fclose(fileID);
% . save black & white image in PNG format to destination folder
% '..\01-Preprocessed_Data\01-HSBW_Camera_Data\01-Pictures\':
file_name = png_files(ii).name;
file_path = fullfile(datasetOutputFolder,folder_010101,...
subfolder_id_temp,file_name);
fileID = fopen(file_path,'w');
fwrite(fileID,file_data,'uint8');
fclose(fileID);
% . read and assign black & white image to Matlab matrix:
bwdata(:,:,ii) = imread(file_path);
end
% ...
end
Doing that, I'm getting the following error randomly (0.5% of processed datasets):
'Error using bwReadout
Invalid fileID for worker. File operations must occur on the worker where the file was opened.
Error in run (line 417)
[status(1,ii),output_id,input_ids] = bwReadout(dataset_name,...'
Can anybody explain what causes the error and knows how to solve it?
Thanks in advance, Martin
0 Comments
Accepted Answer
Harald
on 20 Sep 2023
Hi Martin,
are you using a thread-based or process-based pool? In any case, I would not have expected this to error.
Two ideas for addressing this:
a) Create a separate file ID in each loop iteration, i.e., use fileID(ii) instead of fileID.
b) Move the body of the parfor loop into a function.
I hope that either of these resolves the issue for you.
Best wishes,
Harald
More Answers (0)
See Also
Categories
Find more on Loops and Conditional Statements in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!