Viewing Your Test Results :: Using the Test Results Viewer (SystemTest™)

Learn more about SystemTest

Viewing Your Test Results

On this page…
Reserved Keywords Browsing Results Generating Plots Exploring Plots

On this page…

Reserved Keywords

The Test Results Viewer has several reserved keywords that you cannot use as a test result name or as a derived result name. These keywords are:

time
testrun
testruns
metadata
data

If any of these keywords are used as a test result name, they will be prepended with "st_" when loaded in the Test Results Viewer. If you try to use these keywords as a derived result name in the Test Results Viewer, you will get an error message.

Browsing Results

Viewing Test Results notes that the Test Results Viewer contains a data browser within the Data pane. This area of the viewer is one of the first things you see when the Test Results Viewer opens. It shows you the test variables and test vectors your test used, and it shows information about their values in the Data Statistics area.

These data statistics summarize the values of a test result or test vector across all of the tests. For example, the Throttle demo varies the parameters for mass, damping, and stiffness of a Simulink model. Test vectors vary Simulink block parameters for 90 test iterations, and the SystemTest software saves how these changes affect the position of a simulated throttle opening in the position_sim test result.

If you click position_sim in the Results area of the Data pane, the Data Statistics area shows you a summary of statistical information for all 90 iterations. In this example, you have not defined any constraints on your data, so statistical information for the constrained and unconstrained columns is the same. See Creating and Applying Constraints.

Generating Plots explains how you can further explore your test results.

Generating Plots

The Test Results Viewer has a plotting capability that helps you understand your test results. You can determine how values of different inputs (test vectors) affect the overall test results.

To generate any plot:

Click the button corresponding to the type of plot you want to generate. The plot buttons are below the menu bar. For example, click the Line Plot button. See Choosing a Plot for an explanation of your choices. You can also use the Plots menu to generate plots.

Choose the data to use for your X-axis and Y-axis in the Define Plot pane. For example, select *Auto* from the X Axis list and position_sim from the Y Axis list to show the simulated throttle position trajectories at each test iteration. See Choosing a Plot to understand which data types are available on each axis.

Note Selecting *Auto* when creating a plot means the plot will show the exact number of values in the test vector or result you are plotting. For example if you are plotting a test vector that has 50 values, and you select *Auto* for one of the axes, that axis will display 50 points.

Choose a different plot type if you do not want to use the default. To choose a different plot type:
1. Click Plot type in the Define Plot pane.
2. Click the plot type you want to use. For the Throttle demo example, use the default sine wave.
Click the Plot button. The Test Results Viewer renders a plot based on your selections.

Each line in the plot represents a test iteration. If it appears that there are not as many lines as you had test iterations, it is possible that two or more iterations generated similar enough results that they overlap.

Now you can analyze the plot. To help you with this task, you can:

Explore the plot using the plotting tools available to you as explained in Exploring Plots.
Refine what results are shown in your plot as explained in Refining Your Test Results.

Plotting Grouped Test Vectors

You can plot grouped test vectors on both the X and Y axes of scatter plots. Using grouped test vectors in plot configurations allows you to see the relationship between the grouped vectors.

The following figure shows an example of a test that has two grouped test vectors, TestVector1G and TestVector2G. The scatter plot allows both grouped vectors to be shown in the plot, one on each axis. That is useful for Monte Carlo simulation testing. For example, if you have two vectors that vary the mass of two different components in a model, you could see them in relation to each other.

To plot two grouped test vectors in the viewer:

Select Scatter as your Plot type.
Select the first grouped test vector from the X Axis list.
Select the second grouped test vector from the Y Axis list.
Click the Plot button.

You will see a plot similar to the one shown above in which you can see how the two test vectors relate to each other.

Using Grouped Test Vectors as Distinguishing Variables in Subplots

You can also use grouped test vectors as distinguishing variables in subplots. You can select them in the Subplot drop-down lists next to the labels show and and.

For example, in the example shown above, if a test variable TestVariable1 were plotted against a non-grouped test vector TestVector1Ungrouped, TestVector1G could be used to distinguish the resulting scatter points using different marker colors and TestVector2G could be used to distinguish them by different subplot rows. The grouped test vectors would appear in the subplot drop-down lists to allow this configuration.

Choosing a Plot

There are six types of plots. The line plot, mesh plot, and time series plot types have additional subtypes available. Additionally, the Test Results Viewer has rules for determining which test results you can plot on the X-axis, Y-axis, and Z-axis. These rules vary by plot type. The following table explains these selections:

Plot	Description
Line	Standard line plot of Y versus X. Represents scalar or vector data. The default is a wave line, but you can choose a square line sub type. The following data are allowed on each axis: X — Numeric test vectors Y — Numeric test results
Surf	Wireframe surf plot based on X, Y, and Z coordinates. Optional surface sub type available. The following data are allowed on each axis: X — Numeric test vectors Y — Numeric test vectors Z — Numeric test results
Scatter	Standard scatter plot of X and Y where either axis can have numeric test vectors or numeric test results.
Time Series	Plots time series data Y against time (X is always time). Designed to represent Simulink time series object data. The default is a wave line, but you can choose a square line sub type. See Viewing Simulink Time Series Data for more information about this plot type.
Waterfall	Waterfall plot for vectors or time series. One vector or time series can be displayed on each waterfall plot. The meaning of the X, Y, and Z axes is as follows: X — Is automatically selected to be "Auto" if the Z axes is assigned to a vector-valued test result, or "Time" if Z axes is assigned to a time series test result. Y — You can select either Test Run or Iteration. In the former case, if a test is excluded by application of constraints a gap will appear in the waterfall plot at the Y position corresponding to that test. In the latter case, lines representing the test result displayed on the Z axis are always placed in consecutive Y positions. Z — You can select either a single vector-valued numeric test result or a single time series test result.
Image	Lets you look at individual frames from an image sequence saved during a test iteration. Data must be a supported MATLAB Image format, and must be numeric test results whose size is compatible with an image, namely that: It has three or four dimensions. The third dimension has a length of 1 or 3.

Plot

Description

Line

Standard line plot of Y versus X. Represents scalar or vector data. The default is a wave line, but you can choose a square line sub type. The following data are allowed on each axis:

X — Numeric test vectors
Y — Numeric test results

Surf

Wireframe surf plot based on X, Y, and Z coordinates. Optional surface sub type available. The following data are allowed on each axis:

X — Numeric test vectors
Y — Numeric test vectors
Z — Numeric test results

Scatter

Standard scatter plot of X and Y where either axis can have numeric test vectors or numeric test results.

Time Series

Plots time series data Y against time (X is always time). Designed to represent Simulink time series object data. The default is a wave line, but you can choose a square line sub type. See Viewing Simulink Time Series Data for more information about this plot type.

Waterfall

Waterfall plot for vectors or time series. One vector or time series can be displayed on each waterfall plot. The meaning of the X, Y, and Z axes is as follows:

X — Is automatically selected to be "*Auto*" if the Z axes is assigned to a vector-valued test result, or "Time" if Z axes is assigned to a time series test result.
Y — You can select either Test Run or Iteration. In the former case, if a test is excluded by application of constraints a gap will appear in the waterfall plot at the Y position corresponding to that test. In the latter case, lines representing the test result displayed on the Z axis are always placed in consecutive Y positions.
Z — You can select either a single vector-valued numeric test result or a single time series test result.

Image

Lets you look at individual frames from an image sequence saved during a test iteration. Data must be a supported MATLAB Image format, and must be numeric test results whose size is compatible with an image, namely that:

It has three or four dimensions.
The third dimension has a length of 1 or 3.

Exploring Plots

This section describes the tools the Test Results Viewer makes available to help you understand its generated plots. It contains the following topics:

Plotting Tools describes the tools available to help you examine and understand the contents of a generated plot.
Viewing Individual Iteration Values shows how to focus on specific iteration test results in a plot.
Highlighting Values in Your Plot shows how to distinguish test results in a plot.
Exposing Overlapping Plot Lines explains how you can view individual lines in a plot that shows multiple test result values as the same line.

Plotting Tools

The Test Results Viewer integrates the MATLAB Figure Toolbar, which lets you examine and distinguish the test results shown in your plots. See "Plotting Tools—Interactive Plotting" and "Data Exploration Tools" in the MATLAB Graphics documentation for more information.

In addition, the viewer also supports the desktop arrangement tools available in the MATLAB editor. See "Arranging the Desktop — Overview" in the MATLAB documentation.

The Test Results Viewer adds the following features to the MATLAB Figure Toolbar:

Test run selection — Lets you click different test runs in the plot and see the test vector and test results for that iteration in the Current Iteration pane. Viewing Individual Iteration Values shows an example of how to use this.
Lock the plot — Prevents constraints from changing the test results displayed in the plot.

Viewing Individual Iteration Values

Every test iteration has its own representation in a plot unless you screened it out with a constraint (Refining Your Test Results explains constraints). By clicking a line, marker, or surface in a plot with the test run selection tool, you can see the information associated with that test iteration in the Current Iteration pane.

For example, Generating Plots demonstrates how to generate a plot showing all test iteration results of the Throttle demo. You can use the Test Results Viewer plotting tools to zoom in on areas of the plot and determine which iteration was responsible for the result.

Click the Zoom In button.
Move the mouse pointer over an area of the plot you want to investigate further.
Left-click your mouse or click and drag over the area you want to see. The plot redraws to show this area.

You can repeat zooming in until you have the level of detail you want.
To turn off the Zoom, click the Zoom In button again.
Click the Select an iteration button in the Figure Toolbar.
Click one of the plotted lines in the line plot. The viewer marks the line.

The viewer simultaneously populates the Current Iteration pane with information about the values for all test vectors and test results for your selected test iteration. This lets you easily see what test conditions generated a specific result.

Highlighting Values in Your Plot

The Test Results Viewer lets you further distinguish your test results for any given plot by letting you control how a plot renders the data on each axis. This is useful in deciphering test results on a plot—especially when the initial plot has a large number of test results closely grouped together. This section explains how you use the Define Plot pane to modify the appearance of your plot without modifying the underlying test results. (See Refining Your Test Results for information about modifying the test results used to render a plot.)

The Define Plot pane provides four ways to distinguish plotted test results:

Color
Markers
Subplot rows
Subplot columns

For example, the Throttle demo shows the effect of variations in mass, damping, and stiffness on a component of a Simulink model. The plot you generated in Generating Plots shows test results for of all test iterations, but it is impossible to determine how changes to each test vector affected this outcome. To distinguish the test results on the plot:

Zoom in on an area of the line plot so that you can see individual test iterations (as explained in Viewing Individual Iteration Values).
In the Define Plot pane, click Show > Damping.
Select color from the as list.
Click the Refresh Plot button. The plot lines change to show a range of colors.

You now have some idea how damping has affected the test results. You have a cluster of blue, green, and red indicating that damping is the same value in each cluster, which you can confirm by using the test selection tool to choose lines and by viewing the value for the Damping test vector in the Current Iteration pane.

You can modify the appearance of another set of test vectors to further understand the test results. For example, the menu below Damping can be used to distinguish variations in mass with markers.

Click the menu next to and.
Select Mass from the list.
Click the menu next to as and select marker type.
Click the Refresh Plot button.

The viewer redraws the plot to show markers distinguishing variations in mass. Notice how each cluster of lines has its own unique color and marker, which shows that variations in damping and mass have a visible effect when you run the model.

You can add two more rows using the + button in the Define Plot pane to distinguish your test results further.

Note These colors and markers do not necessarily show the same value throughout the overall plot. The viewer cycles through all colors and markers in the palette making it possible for different test result values to have the same color or marker.

Exposing Overlapping Plot Lines

It is possible for plot lines and points to overlap and appear undistinguishable. When multiple lines overlap, you can create subplots to distinguish the data points.

For example, if you create a line plot for the Throttle demo with the X-axis set to *Auto* and the Y-axis set to position_sim, the Test Results Viewer renders a plot with plot lines in close proximity.

This plot has 90 lines that are too close together to be able to discern clear patterns. You can use the Define Plot pane to distinguish plots of test results by placing the generated lines of a test in individual subplots. Each subplot shows the test vector values associated with the test results being plotted. The number of runs per test vector value determines how many subplots you can generate. Using the Throttle demo, you can generate subplots based on changes in damping, mass, or stiffness. For example, what effect did changes in mass have on these test results? To show its effect: