Model Coverage for Stateflow Charts
How Model Coverage Reports Work for Stateflow Charts
A model coverage report is generated automatically if you simulate your model
using the Run button. If you did not use the
Run button, or you loaded coverage data without
simulating the model, generate a Model Coverage report using
Stateflow® charts, Simulink®
Coverage™ records the execution of the chart itself and the execution of
states, transition decisions, and individual conditions that compose each
decision. After simulation ends, the model coverage reports on how
thoroughly a model was tested. The report shows:
How many times each exclusive sub-state is executed or exited from its parent superstate and entered due to parent superstate history
How many times each transition decision has been evaluated as true or false
How many times each condition has been evaluated as true or false
To measure model coverage data for a Stateflow chart, you must:
Have a Stateflow license.
Have debugging/animation enabled for the chart.
Specify Coverage Report Settings for Stateflow Charts
Specify coverage recording settings from the Coverage pane of the Configuration Parameters dialog box.
Enabling coverage analysis also enables the selection of different coverage metrics. The following sections address only coverage metrics that affect reports for Stateflow charts. These metrics include decision coverage, condition coverage, and MCDC coverage.
Cyclomatic Complexity for Stateflow Charts
Cyclomatic complexity is a measure of the complexity of a software module based on its edges, nodes, and components within a control-flow chart. It provides an indication of how many times you need to test the module.
The calculation of cyclomatic complexity is as follows:
CC = E - N + p
CC is the cyclomatic complexity,
is the number of edges,
N is the number of nodes, and
p is the number of components.
Within the Model Coverage tool, each decision is exactly equivalent to a single control flow node, and each decision outcome is equivalent to a control flow edge. Any additional structure in the control-flow chart is ignored since it contributes the same number of nodes as edges and therefore has no effect on the complexity calculation. Therefore, you can express cyclomatic complexity as follows:
CC = OUTCOMES - DECISIONS + p
For analysis purposes, each chart counts as a single component.
Decision Coverage for Stateflow Charts
Decision coverage interprets a model execution in terms of underlying decisions where behavior or execution must take one outcome from a set of mutually exclusive outcomes.
Full coverage for an object of decision means that every decision has had at least one occurrence of each of its possible outcomes.
Decisions belong to an object making the decision based on its contents or properties. The following table lists the decisions recorded for model coverage for the Stateflow objects owning them. The sections that follow the table describe these decisions and their possible outcomes.
If a chart is a triggered Simulink block, it must decide whether or not to execute its block.
If a chart contains exclusive (OR) substates, it must decide which of its states to execute.
If a state is a superstate containing exclusive (OR) substates, it must decide which substate to execute.
If a state has
If a transition is a conditional transition, it must decide whether or not to exit its active source state or junction and enter another state or junction.
Chart as a Triggered Simulink Block Decision
If the chart is a triggered block in a Simulink model, the decision to execute the block is tested. If
the block is not triggered, there is
no decision to execute the
block, and the measurement of decision coverage is not applicable
Chart Containing Exclusive OR Substates Decision
If the chart contains exclusive (OR) substates, the decision on which
substate to execute is tested. If the chart contains only parallel
AND substates, this coverage measurement is not applicable
Superstate Containing Exclusive OR Substates Decision
Since a chart is hierarchically processed from the top down, procedures such as exclusive (OR) substate entry, exit, and execution are sometimes decided by the parenting superstate.
Decision coverage for superstates applies only to exclusive (OR) substates. A superstate makes no decisions for parallel (AND) substates.
Since a superstate must decide which exclusive (OR) substate to process, the number of decision outcomes for the superstate is the number of exclusive (OR) substates that it contains. In the examples that follow, the choice of which substate to process can occur in one of three possible contexts.
Implicit transitions appear as dashed lines in the following examples.
|Context||Example||Decisions That Occur|
During processing of state
Implicit substate exit
A transition takes place whose source is superstate A and whose destination is state B.
|If the superstate has two exclusive (OR) substates, it is the decision of superstate A which substate performs the implicit transition from substate to superstate.|
Substate entry with a history junction
A history junction records which substate was last active before the superstate was exited.
|If that superstate becomes the destination of one or more transitions, the history junction decides which previously active substate to enter.|
For more information, see State Details Report Section.
State with On Event_Name Action Statement Decision
A state that has an
statement must decide whether to execute that statement based on the
reception of a specified event or on an accumulation of the
specified event when using temporal logic operators.
Conditional Transition Decision
A conditional transition is a transition with a triggering event and/or a guarding condition. In a conditional transition from one state to another, the decision to exit one state and enter another is credited to the transition itself.
Only conditional transitions receive decision coverage. Transitions without decisions are not applicable to decision coverage.
Condition Coverage for Stateflow Charts
Condition coverage reports on the extent to which all possible outcomes are achieved for individual subconditions composing a transition decision or for logical expressions in assignment statements in states and transitions.
For example, for the decision [A & B & C] on a transition, condition coverage reports on the true and false occurrences of each of the subconditions A, B, and C. This results in eight possible outcomes: true and false for each of three subconditions.
For more information, see Transition Details Report Section.
MCDC Coverage for Stateflow Charts
The Modified Condition Decision/Coverage (MCDC) option reports a test's coverage of occurrences in which changing an individual subcondition within a logical expression results in changing the entire expression from true to false or false to true.
For example, if a transition executes on the condition [
C1 & C2
& C3 | C4 & C5], the MCDC report for that
transition shows actual occurrences for each of the five subconditions
C1, C2, C3, C4, C5) in which changing its result
from true to false is able to change the result of the entire condition from
true to false.
Relational Boundary Coverage for Stateflow Charts
If a transition in a Stateflow chart involves a relational operation, it receives relational boundary coverage. For more information, see Relational Boundary Coverage.
Simulink Design Verifier Coverage for Stateflow Charts
You can use the following Simulink Design Verifier™ functions inside Stateflow charts:
If you do not have a Simulink Design Verifier license, you can collect model coverage for a Stateflow chart containing these functions, but you cannot analyze the model using the Simulink Design Verifier software.
When you specify the Objectives and Constraints coverage metric in the Coverage pane of the Configuration Parameters dialog box, the Simulink Coverage software records coverage for these functions.
Each of these functions evaluates an expression
expr, for example,
any valid Boolean MATLAB® expression. Simulink
Design Verifier coverage measures the number of time steps that the expression
expr evaluates to
true for at least one
time step, Simulink
Design Verifier coverage for that function is 100%. Otherwise, the Simulink
Coverage software reports coverage for that function as 0%.
Consider a model that contains this Stateflow chart:
To collect coverage for Simulink Design Verifier functions, on the Coverage pane in the Configuration Parameters dialog box, select Objectives and Constraints.
After simulation, the model coverage report lists coverage for the
Model Coverage Reports for Stateflow Charts
The following sections of a Model Coverage report were generated by simulating
sf_boiler model, which includes the Bang-Bang
Controller chart. The coverage metrics for MCDC are enabled for this report.
Summary Report Section
The Summary section shows coverage results for the entire test and appears at the beginning of the Model Coverage report.
Each line in the hierarchy summarizes the coverage results at that level and the levels below it. You can click a hyperlink to a later section in the report with the same assigned hierarchical order number that details that coverage and the coverage of its children.
The top level,
sf_boiler, is the Simulink model itself. The second level, Bang-Bang Controller,
is the Stateflow chart. The next levels are superstates within the
chart, in order of hierarchical containment. Each superstate uses an
SF: prefix. The bottom level, Boiler Plant model, is an additional
subsystem in the model.
Subsystem and Chart Details Report Sections
When recording coverage for a Stateflow chart, the Simulink Coverage software reports two types of coverage for the chart—Subsystem and Chart.
Subsystem — This section reports coverage for the chart:
Coverage (this object): Coverage data for the chart as a container object
Coverage (inc.) descendants: Coverage data for the chart and the states and transitions in the chart.
If you click the hyperlink of the subsystem name in the section title, the Bang-Bang Controller block is highlighted in the block diagram.
Decision coverage is not applicable (
NA) because this chart does not have an explicit trigger. Condition coverage and MCDC are not applicable (
NA) for a chart, but apply to its descendants.
Chart — This section reports coverage for the chart:
Coverage (this object): Coverage data for the chart and its inputs
Coverage (inc.) descendants: Coverage data for the chart and the states and transitions in the chart.
If you click the hyperlink of the chart name in the section title, the chart opens in the Stateflow Editor.
Decision coverage is listed appears for the chart and its descendants. Condition coverage and MCDC are not applicable (
NA) for a chart, but apply to its descendants.
State Details Report Section
For each state in a chart, the coverage report includes a State section with details about the coverage recorded for that state.
sf_boiler model, the state
On resides in the box
On is a
superstate that contains:
A history junction
The coverage report includes a State section on
The decision coverage for the
On state tests the
decision of which substate to execute.
The three decisions are listed in the report:
Under Substate executed, which substate to execute when
Under Substate exited when parent exited, which substate is active when
NORMis listed as never being active when
Onexits because the coverage tool sees the supertransition from
Offas a transition from
Under Previously active substate entered due to history, which substate to reenter when
Onre-executes. The history junction records the previously active substate.
Because each decision can result in either
NORM, the total possible outcomes are 3 × 2 = 6. The results indicate that five of six possible
outcomes were tested during simulation.
Cyclomatic complexity and decision coverage also apply to descendants
On state. The decision required by the
[warm()] for the transition from HIGH
to NORM brings the total possible decision outcomes to 8. Condition
coverage and MCDC are not applicable (
NA) for a
Nodes and edges that make up the cyclomatic complexity calculation have no direct relationship with model objects (states, transitions, and so on). Instead, this calculation requires a graph representation of the equivalent control flow.
Transition Details Report Section
Reports for transitions appear under the report sections of their owning objects. Transitions do not appear in the model hierarchy of the Summary section, since the hierarchy is based on superstates that own other Stateflow objects.
The decision for this transition depends on the time delay of 40
seconds and the condition
[cold()]. If, after a
40 second delay, the environment is cold (
1), the decision to execute this transition and
turn the Heater on is made. For other time intervals or environment
conditions, the decision is made not to execute.
For decision coverage, both true and false outcomes occurred. Because two of two decision outcomes occurred, coverage was full or 100%.
Condition coverage shows that only 4 of 6 condition outcomes were
tested. The temporal logic statement
after(40,sec) represents two conditions:
the occurrence of
sec and the time delay
after(40,sec). Therefore, three
conditions on the transition exist:
cold(). Since each of these decisions can
be true or false, six possible condition outcomes exist.
The Conditions analyzed table shows each condition as a row with the recorded number of occurrences for each outcome (true or false). Decision rows in which a possible outcome did not occur are shaded. For example, the first and the third rows did not record an occurrence of a false outcome.
The condition varies from true to false.
All other conditions contributing to the decision outcome remain constant.
The outcome of the decision varies from true to false, or the reverse.
For three conditions related by an implied AND operator, these criteria can be satisfied by the occurrence of these conditions.
Notice that in each line, the condition tested changes from true to false while the other condition remains constant. Irrelevant contributors are coded with an "x" (discussed below). If both outcomes occur during testing, coverage is complete (100%) for the condition tested.
The preceding report example shows coverage only for condition 2. The false outcomes required for conditions 1 and 3 did not occur, and are indicated by parentheses for both conditions. Therefore, condition rows 1 and 3 are shaded. While condition 2 has been tested, conditions 1 and 3 have not and MCDC is 33%.
For some decisions, the values of some conditions are
irrelevant under certain circumstances. For example, in the decision
C1 & C2 & C3 | C4 & C5] the
left side of the
| is false if
any one of the conditions
C3 is false. The same applies to the
right side result if either
C5 is false. When searching for matching
pairs that change the outcome of the decision by changing one
condition, holding some of the remaining conditions constant is
irrelevant. In these cases, the MCDC report marks these conditions
with an "x" to indicate their irrelevance as a contributor to the
result. These conditions appear as shown.
Consider the first matched pair. Since condition 1 is true in the
True outcome column, it
must be false in the matching False
outcome column. This makes the conditions
C3 irrelevant for the false outcome since
C1 & C2 & C3 is
always false if
C1 is false. Also, since the false
outcome is required to evaluate to false, the evaluation of
C4 & C5 must also be false. In this
case, a match was found with
C4 = F, making
Model Coverage for Stateflow State Transition Tables
State transition tables are an alternative way of expressing modal logic in Stateflow. Stateflow charts represent modal logic graphically, and state transition tables can represent equivalent modal logic in tabular form. For more information, see State Transition Tables (Stateflow).
Coverage results for state transition tables are the same as coverage results
for equivalent Stateflow charts, except for a slight difference that arises in coverage
of temporal logic. For example, consider the temporal logic expression
after(4, tick) in the Mode Logic chart of the
slvnvdemo_covfilt example model.
In chart coverage, the
after(4, tick) transition represents
two conditions: the occurrence of
tick and the time delay
after(4, tick). Since the temporal event
never false, the first condition is not
satisfiable, and you cannot record 100% condition and MCDC coverage for the
In state transition table coverage, the
transition represents a single decision, with
no subcondition for the occurrence of
tick. Therefore, only decision coverage is
For state transition tables containing temporal logic decisions, as in the above example, condition coverage and MCDC is not recorded.
Model Coverage for Stateflow Atomic Subcharts
In a Stateflow chart, an atomic subchart is a graphical object that allows you to reuse the same state or subchart across multiple charts and models.
When you specify to record coverage data for a model during simulation, the Simulink Coverage software records coverage for any atomic subcharts in your model. The coverage data records the execution of the chart itself, and the execution of states, transition decisions, and individual conditions that compose each decision in the atomic subchart.
model and record decision coverage:
Open the Stateflow example Map Input and Output Data for an Atomic Subchart.
This model contains two Sine Wave blocks that supply input signals to the Stateflow chart. This chart contains two atomic subcharts—A and B—that are linked from the same library chart, also named A. The library chart contains the following objects:
In the Simulink Editor, select Model Settings on the Modeling tab. Select the Coverage pane of the Configuration Parameters dialog box.
Select Enable coverage analysis and then select Entire System.
Click OK to close the Configuration Parameters dialog box.
sf_atomic_subcharts_map_iodata_fixedmodel by clicking the Run button.
When the simulation completes, the coverage report opens.
The report provides coverage data for atomic subcharts A and B in the following forms:
For the atomic subchart instance and its contents. Decision coverage is not applicable (
NA) because this chart does not have an explicit trigger.
For the library chart A and its contents. The chart itself achieves 100% coverage on the input
u1, and 88% coverage on the states and transitions inside the library chart.
Atomic subchart B is a copy of the same library chart A. The coverage of the contents of subchart B is identical to the coverage of the contents of subchart A.
Model Coverage for Stateflow Truth Tables
Types of Coverage in Stateflow Truth Tables
Simulink Coverage reports model coverage for the decisions the objects make in a Stateflow chart during simulation. The report includes coverage for the decisions the truth table functions make.
|For this type of truth table...||The report includes coverage data for...|
Conditions and only those actions that have decision points.
With the MATLAB for code generation action language, you can specify decision points in actions using control flow constructs, such as loops and switch statements.
To measure model coverage data for a Stateflow truth table, you must have a Stateflow license. For more information about Stateflow truth tables, see Obtain Cumulative Coverage for Reusable Subsystems and Stateflow® Constructs.
Analyze Coverage in Stateflow Truth Tables
If you have a Stateflow license, you can generate a model coverage report for a truth table.
Consider the following model.
The Stateflow chart contains the following truth table:
When you simulate the model and collect coverage, the model coverage report includes the following data:
The Coverage (this object) column shows
no coverage. The reason is that
the container object for the truth table function—the
Stateflow chart—does not decide whether to execute the
ttable truth table.
The Coverage (inc. descendants) column shows coverage for the truth table contents. Coverage for the descendants in the Coverage (inc. descendants) column includes coverage for the conditions and decisions of the logical expressions represented by the truth table.
Coverage for the decisions and their individual conditions in the
ttable truth table function are as
Model Coverage Display for Stateflow Charts
Simulink Coverage displays model coverage results for individual blocks directly in Stateflow charts. When you simulate your model with coverage enabled, the model displays:
Highlighting for Stateflow elements that receive model coverage during simulation
A context-sensitive display of summary model coverage information for each object
For details on enabling coverage highlighting, see Enable Coverage Highlighting.
Display Model Coverage with Model Coloring
When you enable coverage and simulate the model with the
Run button, the model highlights
individual Stateflow elements receiving coverage. If you run your model
sim the model does not display coverage
results by default. In this case, you can see the model highlighting
sf_carmodel from Simulate Chart as a Simulink Block With Local Events (Stateflow).
In the Modeling tab, click Model Settings.
In the Coverage pane of the Configuration Parameters dialog box, select Enable coverage analysis.
In the Coverage metrics section, set Structural coverage level to
Modified Condition Decision Coverage (MCDC).
Simulate the model by clicking the Run (Coverage) button.
Open the shift_logic Stateflow chart.
After simulation ends, the model highlights the chart objects that were analyzed for coverage.
The colors indicate the completeness of coverage analysis:
Green border for full coverage
Red border for partial or missing coverage
Light grey for elements not analyzed for coverage
States that include executable code and conditional transitions that use MATLAB as the action language display granular text coloring based on which outcomes are satisfied. Green indicates satisfied outcomes and red indicates unsatisfied outcomes. For example, consider the following chart:
In this example, the
if statement has evaluated to
both true and false and therefore has full decision coverage. Within
the statement, condition
a > 0 evaluated to both
true and false and has full condition coverage. Condition
> 0, however, evaluated to true but not false and
therefore has only partial condition coverage.
Code Coverage for C/C++ code in Stateflow Charts
Simulink Coverage can record code coverage if your Stateflow chart contains custom C/C++ code. For more information, see Coverage for Custom C/C++ Code in Simulink Models.