Contents

Video Stabilization

This example shows how to develop a video application from Simulink® models using IDE Link component.

As a best practice, look at the Video Stabilization Video Stabilization example provided in the Computer Vision System Toolbox™ before working with this workflow example. Reviewing the Video Stabilization example helps you understand the video application this workflow uses and how it was built.

Supported configurations:

Configuration 1:
Configuration 2:

Recommended tutorials:

Models used in this example:

Introduction

You will use IDE Link component to generate complete projects, including application source code and linker command files, from Simulink models. By invoking the Processor-In-the-Loop (PIL) co-simulation feature you will verify the generated code by running it on your processor with a Simulink model serving as a test bench. Similarly, using the Real-Time Execution Profiler you will assess execution performance and behavior of the generated code. Based on the Execution Profiler results, you will regenerate code by selecting various generation settings to improve the execution performance of the generated code.

This example suggests a development workflow for a specific fixed-point video application to be deployed on an embedded processor, but the process may be applied to other applications. The workflow starts with an application designed with a Simulink model and proceeds through generating verified, profiled and optimized code for the selected processor. The example guides you through intermediate stages of verification and profiling by showing the use of PIL, Real-Time Execution Profiling and Stack Profiling.

The tasks are arranged in the suggested order, but are presented in an independent manner that allows you to skip sections as you wish.

Task 1 - Design and Simulate Algorithm in Floating-Point Arithmetic

Typically, you start designing a system using floating-point arithmetic. This lets you concentrate on the basic algorithm elements without worrying about the processor implementation. The goal of this stage is to confirm feasibility of the basic system design. This design activity may involve viewing and recording the system responses to known inputs. Simulink offers many blocks to help with this task, for instance, Scope, Floating Scope, Display, To Workspace, and Frame Rate Display. In this example, you will use Video Viewer to verify that the algorithm behaves as intended when processing a known video sequence.

The following figure shows a model that simulates the stabilization algorithm in floating-point arithmetic. For detailed information on the model, click on the Info button in the lower left corner of the model.

Simulate the algorithm in floating-point arithmetic

1. Open the Floating-Point ModelFloating-Point Model and click the "Run" button to run the simulation.

2. Under "Display" > "Signal & Ports", select "Port Data Types".

3. Observe the data types for different signal lines, now shown in the model.

4. Observe the input and the output video shown on the video display. Notice that the output video has been stabilized by removing the effects of camera horizontal and vertical movements.

5. Stop the simulation by clicking the "Stop" button.

Task 2 - Design and Simulate Algorithm in Fixed-Point Arithmetic

After confirming the feasibility of the basic design using floating-point arithmetic, you will convert the model to fixed-point arithmetic. At this stage, you will model the fixed-point numerical characteristics of your embedded processor. This emulation of fixed-point in the simulation environment requires complex implementation, which leads to a slower simulation time than when using floating-point arithmetic representation. You can accelerate the model simulation by using the Accelerator mode.

The following figure shows a model that simulates the stabilization algorithm in fixed-point arithmetic.

Simulate the algorithm in fixed-point arithmetic

1. Open the Fixed-Point ModelFixed-Point Model and click the "Run" button to run the simulation.

2. Observe the data types for different signal lines. Observe the Frame Rate.

3. Verify that the output video shown on the right side of the video display is still stabilized.

4. Stop the simulation by clicking the "Stop" button.

5. Select Accelerator simulation mode and click the Start button to run the simulation again. Observe higher Frame Rate.

Task 3 - Design and Simulate Algorithm Using Row-Major Video Data

If the model is going to be used for generating code to be executed on an embedded processor, column-major data orientation may not be appropriate for certain processors. Therefore, those blocks in the model whose default orientation is column-major need to be set to row-major. An example of such a block in this model is Insert Text.

The following figure shows a model that simulates the stabilization algorithm in fixed-point arithmetic using row-major data orientation, which is more attuned to the hardware implementation.

Simulate the algorithm in fixed-point arithmetic using row-major data

1. Open the Row-Major ModelRow-Major Model and click the "Run" button to run the simulation.

2. Verify that the output video shown on the right side of the video display is still stabilized.

3. Stop the simulation by clicking the "Stop" button.

4. Open the "Source" block and observe that the Transpose block changes the orientation of the video data to row-major.

5. Open the "Display Results" subsystem. Open the "Insert Text" block and verify that it is set to insert the text while assuming that the input image is transposed.

Task 4 - Generate and Verify Code

After simulating the algorithm and confirming the correctness of the simulation results, the next step is to verify that the generated code behaves the same when run on the embedded processor. PIL simulation can be used to run part of the application on hardware while the rest is run in Simulink.

In this task you will verify the code generated for the "Stabilization" subsystem by making sure that there is no difference between the outputs of the original subsystem and the PIL block representing it. Although you can compare any of the four output signals of the subsystem, for easier viewing, the model has been set to verify Target ROI signal only.

The following figure shows the previous Simulink model with additional blocks used to compare the "Stabilization" algorithm simulation results against a processor-based implementation.

Set up the "Stabilization" subsystem to enable code-generation

1. Open the PIL ModelPIL Model.

2. Open the Model Configuration Parameters dialog and select "Coder Target" under "Code Generation". Click "Target Hardware Resources" and set the parameters to match your target hardware.

For more information on configuring the model for your target hardware see this sectionthis section in the IDE Link documentation. Click OK.

Configure the model to perform PIL simulation

In the following steps, you will configure the model so the build process does the following when you generate code for the "Stabilization" subsystem:

  • Creates a PIL block in a new model

  • Generates the PIL algorithm code and project in the IDE window

  • Builds the generated project

  • Loads the program onto the processor

1. "Select Simulation" > "Model Configuration Parameters" from the model menu.

2. Under "Code Generation" > "Coder Target" > "Tool Chain Automation", set "Build action" to "Create_Processor_In_the_Loop_project".

3. Click "OK" to save the settings.

Create a PIL block for the "Stabilization" subsystem

In the following steps, you will create the PIL algorithm object code and a corresponding PIL block in a new model.

1. Right-click the "Stabilization" subsystem, and select "C/C++ Code" > "Build This Subsystem" from the context menu.

2. Click "Build" button on the opened dialog box. Observe the progress of the build process in the MATLAB® command window.

Copy the PIL block to the model

1. Copy the PIL block to "idelinkvideoworkflowvideo_pil" model, to the location where "Place PIL block here" is written.

2. Connect the signal lines to the corresponding input and output ports of the PIL block

Run PIL simulation

1. Click "Run" button in the toolbar.

2. Compare the simulation and processor implementation results from the "Display" scopes. The difference should be equal to zero.

3. Stop the simulation.

Configure and run TCP/IP-based PIL simulation

If your target platform has Ethernet support, you may be able to use TCP/IP for PIL simulation. TCP/IP-based PIL simulation is generally much faster than the PIL simulation using IDE debugger.

Here are the steps for running TCP/IP PIL simulation:

1. Enable TCP/IP PIL simulation by following the instructions given in Enabling TCP/IP PIL SimulationEnabling TCP/IP PIL Simulation.

2. To start PIL simulation, repeat the steps performed earlier in this task starting with the "Configure the model to perform PIL simulation".

3. Observe the PIL simulation speed. It has significantly increased compared to the PIL simulation using IDE debugger. You can quantify the speed using a "Frame Rate Display" block from the Computer Vision SinksComputer Vision Sinks library.

4. You can also use TCP/IP for Model Block PIL simulation. Follow the steps given earlier to enable TCP/IP. To set up Model Block PIL simulation see: Verifying the Fixed-Point Fuel Control System Verifying the Fixed-Point Fuel Control System

Task 5 - Generate and Profile Code

If you are running generated code on a desktop target or on a processor simulator that does not simulate timers, skip this task.

After verifying the processor implementation results, the next step is to profile the generated code to look for run-time performance improvements.

This task guides you through configuring the model to use the Real-Time Execution Profiler. The profiler generates a graphical display that shows when tasks are activated, preempted, resumed, and completed. In addition, the profiler generates an HTML report with statistical information on each synchronous task.

The following figure shows a Simulink model with additional blocks added to facilitate profiling real-time execution of the generated code.

Configure the model to perform real-time task execution profiling

In the following steps, you will configure the model to generate code that is instrumented to store real-time task profiling statistics that can be analyzed later.

1. Open the Profiling ModelProfiling Model.

2. In your model select "Simulation" > "Model Configuration Parameters".

3. Verify that the "Target Hardware Resources" settings under "Code Generation" > "Coder Target", match your hardware.

4. Check "Profile real-time execution" option. Make sure "Profiled by" is set to "Tasks".

5. Click "OK" to save the configuration and close the dialog box.

Generate HTML report and graphical display of profiling statistics

1. Right-click the "Stabilization" subsystem, and select "C/C++ Code" > "Build This Subsystem" from the context menu.

2. Click "Build" button on the open dialog box to generate, build, load, and execute code on the processor.

3. Let the program run for at least 5 s and then halt it by clicking the "Halt Processor" button in the model.

4. View profiling results by clicking the "View Profiling Results" button in the model.

5. Note the "Average execution time" for the "Stabilization" subsystem.

Task 6 - Optimize Code Using Compiler Options

In this task, you will apply compiler-level optimizations by specifying a compiler switch for function-level optimization. If Real-Time Execution Profiler is available in your configuration, you can assess the execution performance of the optimized code. Otherwise, ignore the parts of this task related to profiling.

The following figure shows the Simulink model that is set for profiling after you turn on compiler optimizations.

Turn on compiler optimizations

1. Open the Optimization ModelOptimization Model.

2. In your model, select "Simulation" > "Model Configuration Parameters".

3. Under "Code Generation" > "Coder Target" > "Tool Chain Automation", enter -o2 in the "Compiler options string" edit box to apply Function optimizations.

4. Click "OK" to save the configuration and close the dialog box.

Generate HTML report and graphical display of profiling statistics

1. Right-click the "Stabilization" subsystem, and select "C/C++ Code" > "Build This Subsystem" from the context menu.

2. Click the "Build" button on the open dialog box to generate, build, load, and execute code on the processor.

3. After a while, halt the process by clicking the "Halt Processor" button in the model.

4. View profiling results by clicking the "View Profiling Results" button in the model.

5. Note the "Average execution time" for the "Stabilization" subsystem and compare it with the one obtained in the previous task.

Task 7 - Optimize Code Using Processor-Specific Intrinsics

If you are running generated code on a desktop target skip this task. If Real-Time Execution Profiler is not available in your configuration, ignore the parts of this task related to profiling.

After verifying and profiling the generated code, the next step is to generate the production code that uses compiler-specific intrinsics to improve the performance of the generated code even further. This is accomplished by automatically replacing standard C constructs for certain numerical operations with the ones using processor and compiler-specific instructions. You will use "Code Replacement Library" feature to specify the replacement scheme that matches your processor family.

The following figure shows the Simulink model that is set for profiling after you turn on CRL optimizations.

Configure and build the model to use processor-specific intrinsics

1. Open the Processor-Specific ModelProcessor-Specific Model.

2. In your model, select "Simulation" > "Model Configuration Parameters".

3. Under "Code Generation" > "Interface", select the "Code Replacement Library" that matches your processor family.

4. Click "OK" to save the configuration and close the dialog box.

Since you are using the same model is in previous step, it is already configured to store real-time task profiling statistics.

Generate HTML report and graphical display of profiling statistics

1. Right-click the "Stabilization" subsystem, and select "C/C++ Code" > "Build This Subsystem" from the context menu.

2. Click "Build" button on the open dialog box to generate, build, load, and execute code on the processor.

3. After a while, halt the process by clicking the "Halt Processor" button in the model.

4. View profiling results by clicking the "View Profiling Results" button in the model.

5. Note the "Average execution time" for the "Stabilization" subsystem and compare it with the one obtained in the previous task.

Task 8 - Profile Stack Usage

To further optimize generated code, you can run the stack profiler to determine your application's stack usage. This is useful for determining the optimum stack size your application needs.

For this task, you will use the handle IDE_Obj saved on the base MATLAB workspace, and the program created from the previous task.

1. Load the program generated from the previous task on the processor.

2. Set up the stack profiler by entering the following command on the MATLAB prompt:

profile(IDE_Obj,'stack','setup')

This fills the processor's stack with a known repeating value, and displays the initial stack usage which is 0%.

3. Run the application. Wait for the application to end, or if it runs indefinitely, stop it after a considerable time has passed.

4. Get a profile of the stack usage by entering the following command on the MATLAB prompt:

profile(IDE_Obj,'stack','report')

Given the results, you either decrease or increase the size you allocated for the stack. You can update the stack size by going to "Simulation" > "Model Configuration Parameters" > "Code Generation" > "Coder Target" > "Tool Chain Automation". Adjust the value written under System stack size (MAUs).

Summary

This example showed a workflow to generate, verify, profile and optimize code for a video application. It used Processor-in-the-Loop, Real-Time Execution Profiler, Code Replacement Library and Stack Profiler features of IDE Link component to accomplish these tasks. Repeat this workflow to achieve the desired results and the level of performance you seek.

Was this topic helpful?