Corner Detection and Image Overlay with Zynq-Based Hardware
This example shows how to use the Computer Vision System Toolbox™ Support Package for Xilinx® Zynq-Based Hardware to target a corner detection algorithm to the Zynq® hardware.
- Computer Vision System Toolbox
- Vision HDL Toolbox™
- HDL Coder™
- HDL Coder Support Package for Xilinx Zynq-7000 Platform
Optionally, to generate, compile, and target a Zynq ARM® software generation model:
- Embedded Coder®
- Embedded Coder Support Package for Xilinx Zynq-7000 Platform
This example follows the algorithm development workflow that is detailed in the Developing Vision Algorithms for Zynq-Based Hardware example. If you have not already done so, please work through that example to gain a better understanding of the required workflow.
This algorithm corresponds to the Vision HDL Toolbox example, Corner Detection. With the Support Package for Zynq-Based Vision Hardware, you get a hardware reference design that allows for easy integration of your targeted algorithm in the context of a vision system.
If you have not yet done so, run through the guided setup wizard portion of the Zynq support package installation. You might have already completed this step when you installed this support package.
On the MATLAB Home tab, in the Environment section of the Toolstrip, click Add-Ons > Manage Add-Ons. Locate Computer Vision System Toolbox Support Package for Xilinx Zynq-Based Hardware, and click Setup.
The guided setup wizard performs a number of initial setup steps, and confirms that the target can boot and that the host and target can communicate.
For more information, see Guided Setup for Vision Hardware.
This model provides a pixel-stream implementation of the algorithm for targeting HDL. Instead of working on full images, the HDL-ready algorithm works on a pixel-streaming interface. This model uses a RGB Resize block that allows the source video frame to be resized for better simulation performance. Alternatively, you may want to perform a crop of the video frame. The blocks in the shaded areas convert to and from pixel stream signals in preparation for targeting.
Video Source The source video for this example comes from either the From Multimedia File block, that reads video data from a multimedia file, or from the Video Capture block, that captures live video frames from an HDMI source connected to the Zynq-based hardware. To configure the source, right-click on the variant selection icon in the lower-left corner of the Image Source block, choose Override using, and select either File or HW.
For this algorithm, the model is configured as listed:
- A pixel format of RGB. This algorithm is written to work on a RGB pixel format, and both the From Multimedia File and Video Capture blocks are configured to deliver video frames in this format. Other supported pixel formats are YCbCr 4:2:2, and Y only.
Algorithm Configuration The algorithm, in addition to processing the image for corner detection has some control as well.
- pbCornerOnly connects to a push button on the board to display the corner detection results without showing the original image.
- cSliceLevel is a configuration option to adjust the corner detection threshold.
- cOverlayColor is a configuration option to adjust the color of the corner markers.
- cOverlayTransp is a configuration option to adjust the blending of the original image and the corner detection image. A value of 255 means full original image and a 0 means full corner detection image.
After targeting, cSliceLevel, cOverlayColor and cOverlayTransp parameters are controllable from the Simulink model through External Mode or Target Hardware execution. pbCornerOnly control port is a pure hardware connection in the targeted design. This port can run at any desired rate including at the pixel clock rate.
However, cSliceLevel, cOverlayTransp and cOverlayColor values are controlled by the embedded processor (or the host in External Mode or Target Hardware mode). Because neither the host nor the embedded CPU can update these controls at the pixel clock rate, a rate on the order of the frame rate is desired. The sample times of the constant blocks attached to these controls are set to execute at the frameSampleTime.
There are two things to note about the simulation outputs:
- During the first frame of simulation output, the Video Display scopes displays a black image. This condition indicates that no image data is available. This behavior is because the output of the pixel-streaming algorithm must be buffered to form a full-frame before being displayed.
- During the second frame of simulation, the Video Display scope for the pixel-streaming output is incorrect with respect to the cSliceLevel and cOverlayTransp parameter values that you configured Simulink model. This discrepancy is because the algorithm uses initial cSliceLevel and cOverlayTransp value of 0. When you use Rate Transition blocks configured in Unit Delay mode the input value is only registered at the output after a full sample period at the input rate. In this case, the input rate corresponds to one video frame. During this initial transition period, the block will output the initial conditions (a value of 0).These blocks are required to ensure a single-rate for all blocks within the subsystem, which is required for HDL code generation. For more information, see Rate Transition.
You can optionally run this simulation without hardware. To modify the frame size for simulation performance, change the Frame size value in the RGB Resize, Frame To Pixels for RGB, and Pixels To Frame for RGB blocks. The lower the frame size, the faster the simulation will run. The minimum frame size is 240p.
Target the Algorithm
After you are satisfied with the pixel streaming algorithm simulation, you can target the pixel algorithm to the FPGA on the Zynq board.
In preparation for targeting, set up the Xilinx tool chain by invoking hdlsetuptoolpath. For example:
>> hdlsetuptoolpath('ToolName','Xilinx Vivado','ToolPath','C:\Xilinx\Vivado\2016.4\bin\vivado.bat');
Execute help hdlsetuptoolpath for more information.
Start the targeting workflow by right clicking the Corner Overlay Algorithm subsystem and selecting HDL Code > HDL Workflow Advisor.
- In Step 1.1, select IP Core Generation workflow and the appropriate platform from the choices: ZedBoard FMC-HDMI-CAM, ZC706 FMC-HDMI-CAM, ZC702 FMC-HDMI-CAM, PicoZed FMC-HDMI-CAM.
- In Step 1.2, select RGB reference design to match the pixel format of the Corner Overlay Algorithm subsystem. Map the other ports of the hardware user logic to the available hardware interface. For this example, map the pbCornerOnly to push button 0. Likewise, map cSliceLevel, cOverlayTransp and cOverlayColor to AXI4-Lite software control.
- Step 2 prepares the design for generation by doing some design checks.
- Step 3 generates HDL code for the IP core.
- Step 4 integrates the newly generated IP core into the larger Vision Zynq reference design.
Execute each step in sequence to experience the full workflow, or, if you are already familiar with preparation and HDL code generation phases, right-click Step 4.1 in the table of contents on the left hand side and select Run to selected task.
- In Step 4.2, the workflow generates a targeted hardware interface model and, if the Embedded Coder Zynq-7000 support package has been installed, a Zynq software interface model. Click Run this task button with the default settings.
Steps 4.3 and 4.4
The rest of the workflow generates a bitstream for the FPGA, downloads it to the target, and reboots the board.
Because this process can take 20-40 minutes, you can choose to bypass this step by using a pre-generated bitstream for this example that ships with product and was placed on the SDCard during setup.
Note: This bitstream was generated with the HDMI pixel clock constrained to 148.5 MHz for a maximum resolution of 1080p HDTV at 60 frames-per-second. To run this example on Zynq hardware with a higher resolution, select the Source Video Resolution value from the drop-down list in Step 1.2.
To use this pre-generated bitstream execute the following:
>> vz = visionzynq(); >> changeFPGAImage(vz, 'visionzynq-zedboard-hdmicam-corner_overlay.bit');
Replace 'zedboard' with 'zc706', 'zc702' or 'picozed' if appropriate.
Alternatively, you can continue with Steps 4.3 and 4.4.
Using the Generated Models from the HDL Workflow Advisor
Step 4.2 generated two, or four, models depending on whether Embedded Coder is installed: A 'targeted hardware interface' model and associated library model, and a 'software interface' model and associated library model. The 'targeted hardware interface' model can be used to control the reference design from the Simulink model without Embedded Coder. The 'software interface' model supports full software targeting to the Zynq when Embedded Coder and the Zynq-7000 (Embedded Coder) support package are installed, enabling External mode simulation, Processor-in-the-loop, and full deployment.
The library models are created so that any changes to the hardware generation model are propagated to any custom targeted hardware simulation or software interface models that exist.
Targeted Hardware Interface Model: In this model, you can adjust the configuration of the reference design and read or drive control ports of the hardware user logic. These configuration changes affect the design while it is running on the target. You can also display captured video from the target device.
Software Interface Model: In this model, you can run in External mode to control the configuration of the reference design, and read or drive any control ports of the hardware user logic that you connected to AXI-Lite registers. These configuration changes affect the design while it is running on the target. You can use this model to fully deploy a software design. (This model is generated only if Embedded Coder and the Zynq 7000 (Embedded Coder) support package are installed.)