Image Category Classification by Using Deep Learning
This example shows you how to create, compile, and deploy a dlhdl.Workflow object with alexnet as the network object by using the Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC. Use MATLAB® to retrieve the prediction results from the target device. Alexnet is a pretrained convolutional neural network that has been trained on over a million images and can classify images into 1000 object categories (such as keyboard, coffee, mug, pencil,and many animals). You can also use VGG-19 and Darknet-19 as the network objects.
Prerequisites
Xilinx ZCU102 SoC development kit
Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC
Deep Learning Toolbox™ Model for Alexnet
Deep Learning Toolbox™
Deep Learning HDL Toolbox™
Load the Pretrained Series Network
To load the pretrained series network alexnet, enter:
snet = alexnet;
To load the pretrained series network vgg19, enter:
% snet = vgg19;To load the pretrained series network darknet19, enter:
% snet = darknet19;To view the layers of the pretrained series network, enter:
analyzeNetwork(snet) % The saved network contains 25 layers including input, convolution, ReLU, cross channel normalization, % max pool, fully connected, and the softmax output layers.
Create Target Object
Use the dlhdl.Target class to create a target object with a custom name for your target device and an interface to connect your target device to the host computer. Interface options are JTAG and Ethernet. To use JTAG,Install Xilinx™ Vivado™ Design Suite 2019.2. To set the Xilinx Vivado toolpath, enter:
% hdlsetuptoolpath('ToolName', 'Xilinx Vivado', 'ToolPath', 'C:\Xilinx\Vivado\2019.2\bin\vivado.bat');
hTarget = dlhdl.Target('Xilinx','Interface','Ethernet');
Create WorkFlow Object
Use the dlhdl.Workflow class to create an object. When you create the object, specify the network and the bitstream name. Specify the saved pretrained alexnet neural network as the network. Make sure that the bitstream name matches the data type and the FPGA board that you are targeting. In this example, the target FPGA board is the Xilinx ZCU102 SoC board. The bitstream uses a single data type.
hW = dlhdl.Workflow('Network', snet, 'Bitstream', 'zcu102_single','Target',hTarget);
Compile the Alexnet Series network
To compile the Alexnet series network, run the compile method of the dlhdl.Workflow object. You can optionally specify the maximum number of input frames.
dn = hW.compile('InputFrameNumberLimit',15) offset_name offset_address allocated_space
_______________________ ______________ _________________
"InputDataOffset" "0x00000000" "12.0 MB"
"OutputResultOffset" "0x00c00000" "4.0 MB"
"SystemBufferOffset" "0x01000000" "28.0 MB"
"InstructionDataOffset" "0x02c00000" "4.0 MB"
"ConvWeightDataOffset" "0x03000000" "16.0 MB"
"FCWeightDataOffset" "0x04000000" "224.0 MB"
"EndOffset" "0x12000000" "Total: 288.0 MB"
dn = struct with fields:
Operators: [1×1 struct]
LayerConfigs: [1×1 struct]
NetConfigs: [1×1 struct]
Program Bitstream onto FPGA and Download Network Weights
To deploy the network on the Xilinx ZCU102 hardware, run the deploy function of the dlhdl.Workflow object. This function uses the output of the compile function to program the FPGA board by using the programming file. It also downloads the network weights and biases. The deploy function starts programming the FPGA device, displays progress messages, and the time it takes to deploy the network.
hW.deploy
### FPGA bitstream programming has been skipped as the same bitstream is already loaded on the target FPGA. ### Deep learning network programming has been skipped as the same network is already loaded on the target FPGA.
Load Image for Prediction
Load the example image.
imgFile = 'espressomaker.jpg';
inputImg = imresize(imread(imgFile), [227,227]);
imshow(inputImg)
Run Prediction for One Image
Execute the predict method on the dlhdl.Workflow object and then show the label in the MATLAB command window.
[prediction, speed] = hW.predict(single(inputImg),'Profile','on');
### Finished writing input activations. ### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 33531964 0.15242 1 33531979 6.6
conv_module 8965629 0.04075
conv1 1396567 0.00635
norm1 622836 0.00283
pool1 226593 0.00103
conv2 3409730 0.01550
norm2 378491 0.00172
pool2 233223 0.00106
conv3 1139273 0.00518
conv4 892869 0.00406
conv5 615895 0.00280
pool5 50267 0.00023
fc_module 24566335 0.11167
fc6 15819119 0.07191
fc7 7030644 0.03196
fc8 1716570 0.00780
* The clock frequency of the DL processor is: 220MHz
[val, idx] = max(prediction);
snet.Layers(end).ClassNames{idx}ans = 'espresso maker'
Run Prediction for Multiple Images
Load multiple images and retrieve their prediction reults by using the mulltiple frame support feature. For more information, see Multiple Frame Support.
The demoOnImage function loads multiple images and retrieves their prediction results. The annotateresults function displays the image prediction result on top of the images which are assembled into a 3-by-5 array.
imshow(inputImg)
demoOnImage;
### Finished writing input activations. ### Running single input activations.
FPGA PREDICTION: envelope FPGA PREDICTION: file FPGA PREDICTION: folding chair FPGA PREDICTION: mixing bowl FPGA PREDICTION: toilet seat FPGA PREDICTION: dining table FPGA PREDICTION: envelope FPGA PREDICTION: espresso maker FPGA PREDICTION: computer keyboard FPGA PREDICTION: monitor FPGA PREDICTION: mouse FPGA PREDICTION: ballpoint FPGA PREDICTION: letter opener FPGA PREDICTION: analog clock FPGA PREDICTION: ashcan
