MATLAB Answers

Wick
6

What, if anything, can be done to optimize performance for modern AMD CPU's?

Asked by Wick
on 20 Apr 2018
Latest activity Edited by Ned Flanders on 4 Dec 2019 at 17:22
I have a AMD Threadripper CPU. I've seen suggestions that AMD-based computers will benefit from changing the math library from Intel's MKL to the AMD Core Math Library (ACML) along with recommendations to change the BLAS.SPEC file. Any threads relating to this seem quite old and it appears that AMD might have changed the name of its math library.
Is it still possible to optimize the math library for AMD's CPUs? If so, what do I need to do?
Thanks!

  21 Comments

Ned - you da man!
Post your solution as an answer and I'll accept it. Put the text from the batch file in your answer so people can roll their own (nobody likes to run random batch files). Also, you might want to take the spaces out of your script name. After I changed it, MATLAB was happier. Nice test script btw.
A couple notes on running this. The shell that executed matlab after the 'call' function remains using the debug cpu. So if you run it once from the batch file it's gone (since the command window disappears). However, if you run the batch file from a prompt, any future executions of Matlab will use the debug cpu. But matlab executed any other way will not. That made side-by-side testing easier.
Any documentation on the various debug cpu extension options? The new TR's have some AVX commands (not the full 512) so it would be nice if there were a debug cpu that supported those.
Thanks again!!!!
Told you! Never give up! hehe
Thanks for the feedback, I reworked the post on reddit accordingly. No download needed anymore. Ideally, if you're happy, use system variables to make the fix permanent. --> see reddit.
So how much do you gain on the TR?
Unfortunately this Debug Mode is undocumented by Intel (for very good reasons i guess... these gangsters)
My gains on TR were very much inline with yours percentage-wise. However, an FFT-heavy code didn't see any speed up. I suspect that one was bound by memory access. However, the matrix multiplication speedup is worth the hack alone! I'm hoping to upgrade to a new TR this season. The memory access should be superior.
Put up an answer so I can accept it. This is worth immortalizing!

Sign in to comment.

3 Answers

Answer by Ned Flanders on 18 Nov 2019 at 0:43
Edited by Ned Flanders on 2 Dec 2019 at 9:21
 Accepted Answer

You should read it if you want to understand the background.
EDIT: Before you start I have a short request for you that you could help me with and serve your own interest. Matlab will not implement this. If you think that Matlab should offer a permanent solution that serves all users independent of whether they use Intel or AMD CPUs, please make a feature request at Matlab to implement a nummeric library (e.g. OpenBLAS) that does not discriminate against non Intel CPUs. Mathworks will not make this change without people advocating for it. Thanks!
tl;dr
Solution 1 (Windows): Create a .bat file using e.g. "Notepad" with the following lines to start Matlab in AVX2 Mode.
@echo off
set MKL_DEBUG_CPU_TYPE=5
matlab.exe
Save as .txt and rename to .bat. If you double-click that file, Matlab will start the MKL in AVX2 Mode. If you start it the normal way, it will remain as always.
You can also download the .bat file from my highdrive if you trust me (which typically you should not as I am a random guy from the internet). If you delete the startup batch file provided in the download or the one you created yourself, its gone and your computer will be as it has been before.
Solution 2 (Windows): If you are happy with the results (which you surely will be :-)), you should make the setting permanent by entering MKL_DEBUG_CPU_TYPE=5 into the System Environment Variables. This has several advantages, one of them being that it applies to all instances of Matlab and not just the one opened using the .bat file.
Doing this will make the change permanent and available to ALL Programs using the MKL on your system until you delete the entry again.
LINUX: (Thanks to foreignrobot)
Simply type in a terminal:
export MKL_DEBUG_CPU_TYPE=5
and then run matlab from the same terminal.
For benchmarking, you can use this script:
To make it permanent in Linux edit your shell's configuration scripts (~/.bashrc for bash, ~/.zshrc for zsh etc) adding the line export MKL_DEBUG_CPU_TYPE=5. That'll apply in any newly opened shell and to apply it in an already open one simply do . ~/.bashrc or whatever your config script name is. (Thanks to lowpolybutt)
P.S.: In case you are on an older AMD FX CPU, you could test whether MKL_DEBUG_CPU_TYPE=4 works for you. This should enable AVX, but I haven't tested this.

  13 Comments

@PatQuillen, thanks I figured that out. The BLAS_VERBOSITY is handy (no spaces allowed in the DOS command I learned). Is there a way to for the BLAS library to reload or do I have to restart MATLAB each time? If you want to stack paths, use a semicolon to separate them (again, in Windows).
I found out how to download a DLL for BLIS but it errors out immediately on "caxpy" (which is pretty basic). So I'm working on it. OpenBlas is next.
@Wick, you have to restart MATLAB every time. There's (currently) no way to force unloading the BLAS.
Valid separators between libraries on the list are space, semicolon, or comma---that's platform independent.
@PatQuillen thanks. That would explain why BLAS_VERSION cannot be set to something within the "Program Files" directory - the default location for MKL. It's breaking the parse at the space in the folder name. I had to copy the mkl.dll to another folder to try setting it as the fall-over library.

Sign in to comment.


Answer by Ken M.
on 9 Nov 2019 at 10:31

You are saying don't use windows for matlab.
Since that is the easy way around, people stop complaining about it after the switch.
Many programs just work faster with linux so if speed is important then linux is an obvious choice also for matlab.

  3 Comments

"Many programs just work faster with linux"
Any evident to back up this claim?
I did a benchmark might be 10 year back and they are comparable.
If you run BENCH on R2019B you'll see the fastest benchmark make by TMW is Windows 7 followed by Linux, though it is not the same HW.
Some compare test from Hardware Unboxed
If you don't mind the change, I wouldn't stay with windows. Overal I think it is a win for everyone. Maybe the main reason is that linux can handle more cores easier.
I also use an astronomy program pixinsight and it also works much faster on linux than windows.
Changing to Linux is not necessarily a possibility in the work flows for many people. Since Windows users on desktops vastly outnumber Linux users, it would be nice if Mathworks could compile with good options for Ryzen/Threadripper/Epyc users.
However, if someone does have the ability to run both operating systems back-to-back, I'd be more than happy to see a comparison of the two addressing per-thread performance.

Sign in to comment.


Answer by Wick
etwa eine Stunde ago
Edited by Wick
41 Minuten ago

The Accepted Answer by @NedFlanders is the best solution so far. Use that one.
But in the comments of that answer, MathWorks staff member @PatQuillen shared some useful undocumented environment variables that allow anyone to hook into any external BLAS library. This has the potential to be very useful for getting other BLAS packages such as OpenBLAS or BLIS working within MATLAB. Those comments are already buried in that thread so I'm pulling them out as a stand-alone solution. I'll repeat the warning from those comments: "This comes with the same caveats that MathWorks has not qualified our products against alternate BLAS implementations, so as above, we can't confirm that using your own BLAS, such as OpenBLAS, will work correctly throughout our products."
We're going to be setting environment variables for the host operating system. This can be done in a temporary or permanent way outside of MATLAB for both Linux and Windows as can be seen in @NedFlander's reddit thread. However, I'm only going to use the setenv command to set these for only one instance of MATLAB.
MATLAB loads the BLAS library the first time it tries to execute a computation so you can make changes before that point. Once the BLAS library is loaded, there's no way to unload it. You'll have to restart.
The environment variable BLAS_VERBOSITY will set MATLAB to detail information to the Command Window about the loading process. A value of 1 is minimal information, 2 is more.
setenv('BLAS_VERBOSITY','1')
The environment variable BLAS_VERSION, if defined, will direct MATLAB to load the BLAS library from the specified file(s) rather than use its built-in default. The files should be the .so/.dll for the BLAS implementation you plan to use that follows the symbols defined by https://netlib.org/blas/.
setenv('BLAS_VERSION','c:\temp\mkl.dll')
Would load a copy of the MKL library copied into c:\temp. You can use a semicolon, comma, or space to join multiple file locations into a single definition of BLAS_VERSION. So if you're writing your own BLAS that doesn't support all the possible methods you could roll over to MKL as a failsafe, e.g.

  0 Comments

Sign in to comment.