number of threads/processes parfor
Show older comments
I am running Matlab (2015a) on our cluster (Linux 64) to run a parameter sweep using a parfor-loop, within which a Fortran program is called. I am running 24 core jobs that spawn up to 80 processes and 600+ threads, which heavily overloads the machines.
How can I control/analyze the number of processes/threads that are started?
I am setting the environment variables MKL_NUM_THREADS, MKL_DOMAIN_NUM_THREADS and MKL_DYNAMIC for each worker within the matlab script by:
pctRunOnAll system('export MKL_NUM_THREADS=1')
pctRunOnAll system('export MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=1,MKL_DOMAIN_BLAS=4"')
pctRunOnAll system('export MKL_DYNAMIC=FALSE')
I set MKL_NUM_THREADS to 1 also in the Fortran program.
Any help is appreciated.
1 Comment
Ryan Povall
on 5 May 2016
Edited: Ryan Povall
on 5 May 2016
Can you provide more information about your workflow? Is this a local cluster? How many workers are you using? How are you measuring these metrics?
The following function seems to be the only thing related to controlling the number of threads that I have found which may or may not be relevant to your use case:
Answers (0)
Categories
Find more on Parallel for-Loops (parfor) in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!