MATLAB Coder: How do I build the Intel MKL-DNN library for Deep Learning C++ code generation and deployment?

I see a few deep learning networks supported for code generation using MATLAB Coder:

• Deep Learning Networks and Layers Supported for C++ Code Generation

I'm looking to generate code from my deep learning network (like AlexNet, GoogLeNet, ResNet, SqueezeNet, VGG-16/19, etc) to run on Intel CPUs using MATLAB Coder and the Intel MKL-DNN library. What are the steps to do this?

Answer 1

[Bill Chou]

Note: The following applies to R2018b, R2019a, and R2019b releases of MATLAB Coder and MKL-DNN v0.14 as described here. For newer releases, see the next answer below.

Background

To generate and run C++ code for Deep Learning, you must have the Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN). Do not use a prebuilt library because some required files are missing. Instead, build the library from source code.

The following describes build instructions for MKL-DNN on Windows and Linux platforms.

Windows MKL-DNN build instructions

To follow these instructions, obtain a recommended compiler and IDE:

• Microsoft Visual Studio 2017
• Intel C++ Compiler 18.0 from Intel Parallel Studio

These instructions assume the use of the Microsoft Visual Studio IDE and compiler. cmake is also required.

1. Download the latest Intel MKL-DNN source code (https://github.com/intel/mkl-dnn/archive/master.zip) or copy source code for a required version for your MATLAB Coder release from https://github.com/intel/mkl-dnn/releases. The version required by the latest MATLAB Coder release is shown in the documentation: https://www.mathworks.com/help/coder/ug/prerequisites-for-deep-learning-with-matlab-coder.html

2. Unzip the source code to a folder, MKLDNN. From the MKLDNN folder, from the Command Prompt, enter:

cd scripts
.\prepare_mkl.bat
cd ..

This creates these libraries at MKLDNN\external\mklml_win_*\lib:

• libiomp5md.dll
• libiomp5md.lib
• mklml.dll
• mklml.lib

3. Run these commands from the MKLDNN folder:

mkdir -p build
cd build
cmake -G “Visual Studio 15 2017 Win64” ..

This creates this Visual Studio solution file in /build:

• Intel(R) MKL-DNN.sln

4. Open the solution file in Visual Studio. Set the Solution Configurations dropdown to Release. Click on Build > Build Solution. This creates these libraries in MKLDNN\build\src\Release:

• mkldnn.dll
• mkldnn.lib

Copy these libraries and the libraries generated in step 2 to C:\Program Files\mkl-dnn\lib.

Note that on Windows® operating systems, special characters and spaces are allowed in the path only if 8.3 file names are enabled. If they are not enabled, replace 'C:\Program Files\mkl-dnn\lib’ with a path that does not include a space. For more information on 8.3 file names, refer to the Windows documentation.

Copy the following files from MKLDNN\include to C:\Program Files\mkl-dnn\include:

• mkldnn.h
• mkldnn.hpp
• mkldnn_debug.h
• mkldnn_types.h

5. Set the MATLAB environmental variable INTEL_MKLDNN to C:\Program Files\mkl-dnn. From the MATLAB Command Window, enter:

setenv('INTEL_MKLDNN', 'C:\Program Files\mkl-dnn\')

Add C:\Program Files\mkl-dnn\lib to the PATH variable.

setenv('PATH', [getenv('INTEL_MKLDNN') filesep 'lib' pathsep getenv('PATH')])

6. If you have not done so already, you should also set the environment variable for Windows. From the Windows Command Prompt, enter:

set PATH=%PATH%; C:\Program Files\mkl-dnn\

Linux MKL-DNN build instructions

To follow these instructions, obtain a recommended compiler:

• GNU g++ with C++11 support

cmake is also required.

1. Download the latest Intel MKL-DNN source code (https://github.com/intel/mkl-dnn/archive/master.zip) or copy source code for a required version of your MATLAB Coder release from https://github.com/intel/mkl-dnn/releases. The version required by the latest MATLAB Coder release is shown in the documentation: https://www.mathworks.com/help/coder/ug/prerequisites-for-deep-learning-with-matlab-coder.html

2. Unzip the source code to a folder, MKLDNN. From the MKLDNN folder, from the Command Prompt, enter:

cd scripts
chmod +x prepare_mkl.sh
./prepare_mkl.sh

This creates these libraries at MKLDNN/external/mklml_lnx_*/lib:

libiomp5.so
libmklml_intel.so.

3. Run these commands from the MKLDNN folder:

mkdir -p build
cd build
cmake ..
make

4. This creates these libraries under MKLDNN/build/src:

1. libmkldnn.so
2. libmkldnn.so.0
3. libmkldnn.so.0.14.0.

Copy these libraries and thelibraries generated in step 2 to /usr/local/mkl-dnn/lib.

Copy the following files from MKLDNN/include to /usr/local/mkl-dnn/include:

• mkldnn.h
• mkldnn.hpp
• mkldnn_debug.h
• mkldnn_types.h

5. Set the MATLAB environmental variable INTEL_MKLDNN to /usr/local/mkl-dnn. From the MATLAB Command Window, enter:

setenv('INTEL_MKLDNN', ‘/usr/local/mkl-dnn')

Add /usr/local/mkl-dnn/lib to the PATH variable:

setenv('LD_LIBRARY_PATH', [getenv('INTEL_MKLDNN') filesep 'lib' pathsep getenv('LD_LIBRARY_PATH')]);

If you have not done so already, you should also set the environment variables for Linux. Use Linux syntax in the Linux terminal to set the variable LD_LIBRARY_PATH to /usr/local/mkl-dnn/lib, and the environment variable INTEL_MKLDNN to /usr/local/mkl-dnn.

[Mark Brinton]

Very helpful. Thanks!

[Usatyuk Vasiliy]

Bill Chou, Thank you!

Answer 2

[Bill Chou]

Note: The following applies to R2020a and newer releases of MATLAB Coder and MKL-DNN v1.0 as described here.

Note: Intel recently renamed the library from MKL-DNN to oneDNN, so we use MKL-DNN and oneDNN interchangeably.

Background

To generate and run C++ code for Deep Learning, you must have the Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN). Do not use a prebuilt library because some required files are missing. Instead, build the library from source code.

The following describes build instructions for MKL-DNN on Windows and Linux platforms.

To build the Intel MKL-DNN library from source code, you must have:

• Operating system with Intel 64 architecture support
• C++ compiler with C++11 standard support
• CMake 2.8.11 or later version

Windows MKL-DNN build instructions

C++ Compiler requirements :

• Microsoft Visual C++ 14.0 (Visual Studio 2015 Update 3)
• If you face compilation errors while building MKL-DNN, please use MSVC Compiler version 19.16.xx or newer

1. Download the mkldnn v1.0 source code from the link (https://github.com/oneapi-src/oneDNN/archive/v1.0.zip) and extract the source code. This action creates the folder oneDNN-1.0.

2. Generate a Microsoft Visual Studio solution “Intel(R) MKL-DNN.sln” by running these commands in command line at oneDNN-1.0 folder:

mkdir build
cd build
cmake -G "Visual Studio 15 2017 Win64" ..

3. Run below command in command line at oneDNN-1.0\build folder:

cmake --build . --config Release

This action creates these libraries in the folder oneDNN-1.0\build\src\Release:

• mkldnn.lib
• mkldnn.dll

Copy these generated libraries to C:\Program Files\mkl-dnn\lib.

Note that on Windows operating systems, special characters and spaces are allowed in the path only if 8.3 file names are enabled. If they are not enabled, replace C:\Program Files\mkl-dnn\lib with a path that does not include a space. For more information on 8.3 file names, refer to the Windows documentation.

Copy the include files from oneDNN-1.0\include and oneDNN-1.0\build\include to C:\Program Files\mkl-dnn\include.

4. Set the MATLAB environment variable INTEL_MKLDNN to C:\Program Files\mkl-dnn. At the MATLAB command prompt, enter:

setenv('INTEL_MKLDNN', 'C:\Program Files\mkl-dnn')

Add C:\Program Files\mkl-dnn\lib to the PATH variable.

setenv('PATH', [getenv('INTEL_MKLDNN') filesep 'lib' pathsep getenv('PATH')])

5. If you have not done so already, you must set the environment variable for Windows. At the Windows Command Prompt, enter:

set PATH=%PATH%; C:\Program Files\mkl-dnn\lib

Also create and set the INTEL_MKLDNN environment variable in windows that point to C:\Program Files\mkl-dnn

Linux MKL-DNN build instructions

C++ Compiler requirements :

• Install any one of the GNU compiler versions from GNU Compiler Collection 4.8, 5.4, 6.1, 7.2, and 8.1

1. Download the mkldnn v1.0 source code from the link (https://github.com/oneapi-src/oneDNN/archive/v1.0.tar.gz) and extract source code by running this command at the terminal:

tar -xvzf oneDNN-1.0.tar.gz

This action creates the folder oneDNN-1.0.

2. To generate the makefile for compilation, run these commands at the terminal from the oneDNN-1.0 folder:

mkdir -p build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..

3. To build the library, run this command at the terminal from the oneDNN-1.0/build folder:

make -j

These commands creates these libraries in the folder oneDNN-1.0/build/src:

• libmkldnn.so
• libmkldnn.so.0
• libmkldnn.so.1.0.0.0

Copy these libraries to /usr/local/mkl-dnn/lib.

Copy the include files from oneDNN-1.0/include and oneDNN-1.0/build/include to /usr/local/mkl-dnn/include.

4. Set the MATLAB environment variable INTEL_MKLDNN to /usr/local/mkl-dnn. At the MATLAB command prompt, enter:

setenv('INTEL_MKLDNN', ‘/usr/local/mkl-dnn')

5. Add /usr/local/mkl-dnn/lib to the PATH variable:

setenv('LD_LIBRARY_PATH', [getenv('INTEL_MKLDNN') filesep 'lib' pathsep getenv('LD_LIBRARY_PATH')]);

If you have not done so already, you must set the environment variables for Linux. Use Linux syntax in the Linux terminal to set the variable LD_LIBRARY_PATH to /usr/local/mkl-dnn/lib, and the environment variable INTEL_MKLDNN to /usr/local/mkl-dnn.

macOS MKL-DNN build instructions (for R2020b and newer)

C++ Compiler requirements:

• Install supported Xcode versions mentioned in https://www.mathworks.com/support/requirements/supported-compilers.html

1. Download the mkldnn v1.0 source code from the link and extract source code by running this command at the terminal:

tar -xvzf mkl-dnn-1.0.tar.gz

This action creates the folder mkl-dnn-1.0.

2. macOS clang compiler does not ship with OpenMp. To have an OpenMp enabled mkldnn library, we need to install brew,libomp

a. Install home brew in host mac PC with below command using bash terminal

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

b. Install OpenMp using brew with command

$(BREW_INSTALL_PATH)/brew install libomp

3. To generate the makefile for compilation, run these commands in the terminal from the mkl-dnn-1.0 folder:

mkdir -p build
cd build
$(CMAKE_INSTALL_PATH)cmake .. -DOpenMP_CXX_FLAGS="-Xclang -fopenmp -I$(OMP_INSTALL_PATH)/include" -DOpenMP_C_FLAGS="-Xclang -fopenmp -I$(OMP_INSTALL_PATH)/include" -DOpenMP_CXX_LIB_NAMES=libomp -DOpenMP_C_LIB_NAMES=libomp -DOpenMP_libomp_LIBRARY=$(OMP_INSTALL_PATH)/lib/libomp.dylib -DCMAKE_SHARED_LINKER_FLAGS="-L$(OMP_INSTALL_PATH)/lib -lomp"

These commands creates these libraries in the folder mkl-dnn-1.0/build/src:

• libmkldnn.dylib
• libmkldnn.0.dylib
• libmkldnn.1.0.0.0.dylib

Copy these libraries to /usr/local/mkl-dnn/lib.

Copy the include files from mkld-dnn-1.0/include to /usr/local/mkl-dnn/include.

4. Copy the OpenMp dependencies required by mkldnn

• OpenMP libraries from $(OMP_INSTALL_PATH)/lib to /usr/local/mkl-dnn/lib
• OpenMP include files from $(OMP_INSTALL_PATH)/include to /usr/local/mkl-dnn/include

5. The OpenMp library path is hardbinded to mkldnn library. Below command would list out this aspect

otool -l libmkldnn.dylib

6. This hard binding of the paths must be changed for portability to other mac machines. Use the below command

install_name_tool -change /usr/local/opt/libomp/lib/libomp.dylib @rpath/libomp.dylib libmkldnn.0.dylib
install_name_tool -id "@rpath/libomp.dylib" libomp.dylib

Changing to rpath would enable portability of mkldnn/OpenMP libraries to different mac machines. This avoids any runtime linking issues.

7. Set the MATLAB environment variable INTEL_MKLDNN to /usr/local/mkl-dnn. At the MATLAB command prompt, enter:

setenv('INTEL_MKLDNN', ‘/usr/local/mkl-dnn')

[JJ L]

Hello, Bill Chou,

I am using the matlab 2020b, and I just installed the Visual Studio 2019 and Intel Parallel Studio XE.

With in the folder of oneDNN-1.0, when I run the commands, the "cmake" cannot be found, so I cannot start your process at all. What should I do?

mkdir build

cd build

cmake -G "Visual Studio 15 2017 Win64" ..

Thank you very much!

[Jaya Shankar]

Hi JJ,

cmake needs to be run as a system command. Please run the cmake commands from your Windows terminal instead of running them within MATLAB.

Jaya

[JJ L]

Thank you, I will try

[Oleg Kyrmyzy]

Is there a mistake? In C++ Compiler requirements you specified "Microsoft Visual C++ 14.0 (Visual Studio 2015 Update 3)", but in step two you wrote "cmake -G "Visual Studio 15 2017 Win64" ..".

Should be "cmake -G "Visual Studio 14 2015 Win64" .."

[Praveen Kumar Gajula]

Hi oleg,

Please refer to this link for desired compiler. The minimum requirement that they are mentioning as per link.

Try out the msvs version mentioned in the 1st link above. Let us know if you face any issues with that.

Thanks,

Praveen

[Bonan Yu]

I have a question about the installation guide MATLAB 2020a in windows:

all built library files in fold oneDNN-1.0. should be copied to C:\Program Files\mkl-dnn

So what I supposed to do is to create mkl-dnn/lib and mkl-dnn/include files then copy the files built in oneDNN-1.0 seperately to mkl-dnn/lib and mkl-dnn/include? However it doesn't work (errors below, it seems like mkl-dnn wasn't correctly installed), maybe because I used MATLAB 2020b? I'm installing 2020a...

Making simulation target "deployment_C_2_sfun", ... 
    
error: 'agentData1' does not name a type 
   agentData1.mat_policy_9_1_1_1_1_1_1_1_ anchor; 
   ^~~~~~~~~~ 
 
gmake: *** [deployment_C_2_sfun.obj] Error -1
Component:Make | Category:Make error
Unable to create mex function 'deployment_C_2_sfun.mexw64' required for simulation.
Component:Simulink | Category:Model error

I also checked my mkl-dnn files installed in anaconda3, there are also bin flies and LICENCE! So is there something wrong in my installation???

--------------------------------------------------------------------------------------------------------------------------------------------------

mkl-dnn in Anaconda3

Thanks in advance !!!

[Praveen Kumar Gajula]

Hi Bonan,

In 20a and 20b we use the same oneDNN1.0 version. There might be conflict of presence of 2 mkldnn libraries installed . Are these 2 paths set as part of environement vairiable. If so, then try to remove anaconda related path from environemnt vairiables and try out. That should solve your problem.

Thanks,

Praveen.

Answer 3

[Xiaoqi]

Hi, Bill

I believe your steps need to be updated, because the mkl-dnn library in the github is changed. I can not make it work by following your steps.

Thanks

Charlie

[Praveen Kumar Gajula]

Hi Xiaoqi,

As per the R2019b pre-requisites documentation we use the mkldnn v0.14 . The above build steps will suffice for the mkldnn v0.14.

We will update the steps as you mentioned once we support latest mkldnn version.

Thank you,

Praveen.

[Bill Chou]

Please see the updated instructions for MKL-DNN v1.0 below.

Answer 4

[Sviatoslav Klos]

I have such an error: Undefined function or variable 'IsAvx512Proc'.

I thought that if my Intel proc does not support Avx512 then it would just use Avx2. Does it make sense? Had someone try MKL-DNN with Intel i5-8265u?

Answer 5

[Praveen Kumar Gajula]

Hi Sviatoslav,

We are sorry to hear that you are facing issues.

You are right that if Intel proc doesnot support Avx512 mkldnn should use Avx2.

And processor i5-8265u should be able to use mkldnn with out any issues.

There is a package installer issue from our end , where it didnot ship the 'IsAvx512Proc' method. Hence you are seeing this error . Could you let me know what is the MATLAB version you are using and did you do any recent upgrade to it.

Thank you,

Praveen.

[Sviatoslav Klos]

Hi Praveen Kumar Gajula,

Thank you for yout response.

Such issues were observed while using R2019a. I tried 2019b and it works there.

P.S. It would be nice if it would be possible to generate ScalingLayer using codegen. In addition, it would be nice if it would be possible to export TanhLayer and ScalingLayer to ONNX format.

[Jaya Shankar]

Hi Sviatoslav,

Deep learning toolbox has a few scaling layer implementations. For instance , there is one implemented for inceptionV3 networks and there is a scaling layer for reinforcement learning toolbox. Can you clarify which one you are using ?

Note that codegen is supported for the scaling layer in inceptionV3 networks since 19a.

Thanks

Jaya

[sara alamri]

didn't work for me !

need help quick please

[Tohru Kikawada]

I think you need to set up the C++ compiler before you build. Please try mex -setup c++. The following link explains for more details.

https://www.mathworks.com/help/matlab/matlab_external/choose-c-or-c-compilers.html

Answer 6

[Praveen Kumar Gajula]

Hi ,

Error is not clear from this image. Could you share us Error report.

Could you also share the MATLAB release details and example link that you are trying out.

Thank you,

Praveen.

[sara alamri]

here is the report of the error in the attachment

i use Matlab R2019b

i also tried to this instructions to install mkl dnn

https://docs.microsoft.com/en-us/cognitive-toolkit/setup-buildmkldnn-vs17

and your instruction but didn't work

i also tried all example in matlab website that related to YOLO code genrator.. all same error

all i want from this is to convert my code"which contain a yolo detector" to C so i can combine it with an android app.

Answer 7

[Praveen Kumar Gajula]

The compilation errors are becuase of mkldnn.hpp file not found.

" c:\users\witch\documents\matlab\examples\r2019b\deeplearning_shared\yolov2objectdetectionusingintelmkldnnexample\codegen\mex\yolov2_detection\MWCNNLayerImpl.hpp(10): fatal error C1083: Cannot open include file: 'mkldnn.hpp': No such file or directory

"

Could you check if you setup the INTEL_MKLDNN path correctly as mentioned in step -

setenv('INTEL_MKLDNN', 'C:\Program Files\mkl-dnn\')

Also could you make sure that there are no spaces in the folders

1. Where you are trying codegen.
2. Where MKLDNN libraries reside.

Thank you,

Praveen.

[sara alamri]

the file is already exist

and there is no space in both paths

I FIX this bug by move this file into project folder

but another error happend : fatal error LNK1181: cannot open input file 'lib\mkldnn.lib'

ninja: build stopped: subcommand failed.

my CPU is intel core i7 7th Gen 2.7 GHz

[sara alamri]

now this error

compilation aborted for MWElementwiseAffineLayer.cpp (code 4)

.....

catastrophic error: cannot open source file "mkldnn.hpp"

...

compilation aborted for MWExponentialLayer.cpp (code 4)

ninja: build stopped: subcommand failed.

[Praveen Kumar Gajula]

We tried the same example and found the same log as yours if we donot set the INTEL_MKLDNN path.

Currently you are seeing the log similar to log present in withOutMKLDNNPath.txt where the INTEL_MKLDNN is not set in the MATLAB. If you set it appropriately you should see log similar to withMKLDNNPATH

Thank you,

Praveen.

Answer 8

[sara alamri]

now i had this error in the attachment using matlab R2019b

and this error using matlab R2019a

??? Error generating code for network yoloV2CatDetector_0. Code generation for
YOLOv2TransformLayer yolov2Transform is not supported for mkldnn target. See documentation
for a list of supported layers with each target.

i use mkldnn V0.14

i also installed minGW v17 which i belive it supports openMP

[Praveen Kumar Gajula]

We donot support the yolov2 for R2019a release.

Regarding the errors in R2019b ,we are unable to reproduce .

It would be great if you could forward the steps that you have used for codegen. This will help us to corner out if there are any steps that are missed out.

Than kyou,

Praveen.

Answer 9

[Mahmoud Abdulsalam]

I have generated a source code for deep learning and found a lot of files in codegen folder. How do I go about integrating it and compile it in Linux.Do I need to make changes to the main.cpp? Because I saw some instructions within the main file asking me to change variable size function argument.

[Hanumanth Hanumantharayappa]

Hi Mahmoud,

Can you take a look at this below documentation? Let me know if this helps.

https://www.mathworks.com/help/coder/ug/generate-an-example-cc-main-function.html?searchHighlight=GenerateExampleMain&s_tid=doc_srchtitle

https://www.mathworks.com/help/coder/ug/structure-of-example-cc-main-function.html

https://www.mathworks.com/help/coder/ug/standalone-c-c-executables-from-matlab-code.html#bsx_bv5

Hanumanth