However, I have tried to use it in my code in the following way and it resulted in an increase of the simulation running time compared to the non-parallelized version of the code.
I am dealing with large arrays and data files and have to calculate some statistics for a very large number of different cases.
In the initial non-parallelized version of my code, I calculate the statistics for case 1 to case 10^7 inside a for loop.
... "read many files"
... "generate large arrays"
... "Calculate statistics"
... "Write statistics in a text file"
In the parallelized version of the code, I use a PARFOR loop. However, I cannot have all the codes to calculate the statistics directly inside the PARFOR loop due to Matlab restrictions and errors. So, I had to create a new function called STAT and copy all the codes to calculate the statistics in this function.
The problem with my parallelized version of the code is that it takes much longer than the non-parallelized version. Inside the PARFOR loop, I have to call a function (STAT) and pass many large arrays (w1.Value) at each iteration.
Does anyone know what is the best way to optimize/parallelize this code?
Works well, thanks!
It does consume a fair bit amount of memory though. Since "random access constant data" seems to be the most common model for many data-intensive applications, it would be great if Matlab somehow supports read-only shared memory across labs in the future.
22 Jan 2014
Worker Object Wrapper
Simplifies managing resources such as large data within PARFOR loops and SPMD blocks