Data on C drive during batch processing

Hello,

My goal is to perform Monte Carlo study (100 000 iterations). We need to run 4 Simulink models per one iteration. We have decided to use Parallel Computiung Tooblox and Batch Processing to split up the study for 8 jobs on two work stations (4 jobs per one work station). I am not able to perform this study beacase MATLAB probably saves some data on C drives in both computers. I thought that the root causes were those *.dmr files in AppData\Local\Temp so I have used such code like:

Simulink.sdi.setAutoArchiveMode(false);

After each iteartion the size *.dmr files do not increase beacuase after each time when we run Simulink we use:

Simulink.sdi.clear;

So, I though that our issue was solved. I have started these 8 jobs once again. Right now I can see that free space on C drives in both work stations continuously decrease. I do not have any idea why. The size of files in location AppData\Local\Temp is constant - 227 megabytes, however on C drive there is more and more data. When we hit 0 bytes of free space on C drive we can not generate any more oupput files out of our scripts and models. This "magic" data stored on the C drive can be counted in gigabytes.

In order to create a job we use:

job_i = batch('ScriptName', 'CaptureDiary', false, 'CurrentFolder', 'D:\...');

where "i" is the number of job (of course, in this case from 1 to 8).

What files are generated on the C drive during jobs execution? How to prevent from saving them? We do not need them.

We use MATLAB R2019b (on first work station) and MATLAB R2021a (on second work station). We have the same issue with unnecessary data stored on C drives in both wotk statations. These unnecessary files take up gigabytes. We can not finish our Monte Carlo study...

[Raymond Norris]

I'm not a Simulink expert, so I'll make some general comments.

• You could probabaly create an array of job objects, as such

for idx = 1:8
    job(idx) = batch('ScriptName','CaptureDiary',false,'CurrentFolder','D:\...');
end

Somewhere, there must be something that differentiates one job from other, perhaps it's ScriptName?

scripts = {'ScriptName_1','ScriptName_2', ...};
for idx = 1:8
    job(idx) = batch(scripts{idx},'CaptureDiary',false,'CurrentFolder','D:\...');
end

• I would suggest writing ScriptName as a function and then calling batch. This way, you don't copy the local workspace to each worker, but instead bound what data is getting passed back and forth.

scripts = {@ScriptName_1,@ScriptName_2, ...};
for idx = 1:8
    job(idx) = batch(scripts{idx},'CaptureDiary',false,'CurrentFolder','D:\...');
end

Obviously, this isn't enough, you'll have to actually modify ScriptName_1, etc. to be functions, which might impeed how you typically call them.

• Particularly for now, I would suggest turning on Diary so that it's easier to debug any issue you're running into. This way you can call

j.Diary

• I'm a bit surprised that you're not seeing a warning that states MATLAB can't change directories to D:\, but maybe that's not an issue. I'm assuming D is local and not a shared network drive? If it's shared, I would suggest replacing it with the UNC path.
• I would suggest having the same versions of MATLAB on both workstations. I suspect somewhere down the road that'll come back to bite you, particulary with Simulink (that is, you won't understand why one feature exists in a job that isn't supported in another).

[Kamil Zajac]

Hello Raymond,

I did what you have suggested. I have changes my scripts to be functions. Then I have created array of job objects. To create a job object I used a code like:

job(1) = batch(@ScriptName, 1, {}, 'CaptureDiary', false, 'CurrentFolder','D:\...');

I am still loosing free space on C drive. It can be counted in gigabytes, so it is not possible to perform the study beacuse, of course, my free space on C drives is limited.