Why are there very large .DMR files filling up my hard drive TEMP directory when running "parsim" or "batchsim"?

67 views (last 30 days)
I am using the Parallel Computing Toolbox to run some Simulink simulations in parallel with "parsim" or "batchsim" and at some point, the hard drive where my pool is located is filled with files in the "TEMP"/"TMP" directory. The files have the extension DMR (.dmr). Is there something I can do to prevent this?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 8 Jan 2024
Edited: MathWorks Support Team on 8 Jan 2024

What is a DMR file?

When you simulate a model that logs signal data, the signal data is streamed to disk in the DMR files that you see in the temporary directory. In parsim/batchsim, each worker is given its own DMR file to store logged data from simulations. 

Parsim and Batchsim Automatic DMR Cleanup

Setup and cleanup of the DMR file is handled by parsim and batchsim automatically. Here are the three main ways the cleanup is handled:
  1. DMR cleared after each simulation (22b and later): If the size of the DMR is above 10 MB, then it is cleared after the simulation is over. The data can be deleted because it is sent back to the client at the end of each simulation. This feature was added in 22b.
  2. DMR cleared at the end of parsim/batchsim: At the end of the simulations, the DMR file is cleared of data automatically.
  3. DMR deleted upon close of pool: The DMR files for workers are deleted when pool is closed. 

Estimate Max DMR Size

Based on the description above, we can approximately calculate the maximum possible DMR size needed during any point during the call to parsim/batchsim.
For R2022b and later:
size_max ≈ N_workers * size_simData
For R2022a and earlier:
size_max ≈ N_simulations * size_simData
where:
  • size_max = approximate max size of DMR files on disk
  • size_simData = size in bytes of amount of logged data. Equal to number of logged signals, times number of time steps logged, times size of datatype in bytes
  • N_workers = number of workers in pool
  • N_simulations = number of simulations in call to parsim/batchsim

Strategies to Reduce DMR Size

If the above automatic cleanup strategies seem to be working but you are still having issues with DMR file size, consider the following strategies:
  1. Reduced amount of logged data: The size of the DMR file is relative to the amount of data being logged in a simulation. Consider using some of the strategies from the doc page below to limit the amount of data your simulation logs.https://www.mathworks.com/help/releases/R2023a/simulink/ug/limit-size-of-logged-data.html
  2. Manually clear DMR after each simulation (22a and earlier): In 22b and later, DMRs are cleared after each simulation. You can manually do something similar in earlier releases by calling Simulink.sdi.clear  in a postSimFcn.
  3. Delete DMR files after worker crash: If the workers crash for any reason during the execution of parsim/batchsim, the automated cleanup code will not be executed, and you may need to manually clear the temp directory if the files are very large. If MATLAB is closed, these files can be safely deleted.

Example on Local Pool

The plot below shows the size of the logged data file (DMR) for a 4-worker local pool across 9 runs each. The amount of data logged during earlier simulations is more than that logged during later simulations. You can see the slope for the change in the size of the DMR file decreases across these runs. This demonstrates the effect of reducing the data logging requirements of your simulation. Additionally, in R2022b you can see that once the DMR size passes 10 MB, it is automatically flushed of redundant data between simulations (after simulation 3). "END" shows how all the data is cleared during the cleanup stage of "parsim" and "CLOSE" shows how the files are deleted upon the pool being closed.
 

More Answers (0)

Categories

Find more on Run Multiple Simulations in Help Center and File Exchange

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!