polyspace.test.CoverageResults Class

Namespace: polyspace.test

(Python) Review code coverage results

Since R2024b

expand all in page

Description

This Python® class contains code coverage results obtained from executing C/C++ tests.

Creation

Syntax

coverageResults = profilingResults.Coverage

Description

coverageResults = profilingResults.Coverage loads coverage results:

Properties

expand all

List of files associated with the coverage results, specified as a list of polyspace.test.CoverageFile objects with the following properties:

PropertyDescription
FullPathFull path to file
Path

Shortest unique file path. This is the shortest path to the file that allows it to be distinguished from another file in the list with the same name.

For instance, suppose there are two files with the same name on these paths:

/common/folder/a/path/to/file1.c
/common/folder/other/path/to/file1.c
The shortest unique paths corresponding to these full paths are:
a/path/to/file1.c
other/path/to/file1.c

Functions

Functions in the file, specified as a list of polyspace.test.ProfilingFunction objects with these string properties:

  • File: Full path to file

  • Function: Name of function

To drill down on profiling results for a specific metric and file, use the getCoverageInfo() method. See Methods.

Whether compact mode was enabled for code coverage calculation, specified as a boolean. The compact mode is disabled by default. To enable this mode, set the CoverageOptions.Compact option in the polyspace.project.TestConfig object associated with your project.

Coverage filters associated with the coverage results object, specified as a polyspace.test.CoverageFiltersView object. To inspect the filter rules, index into the Rules property of this polyspace.test.CoverageFiltersView object. To modify the list of filters associated with the coverage results object, use the importFilters and resetFilters methods documented below.

Methods

expand all

Examples

collapse all

This example shows how to see an overview of code coverage results after test execution.

In general, you generate and manage Polyspace Test™ results by using classes from the polyspace.project and polyspace.test modules. Before starting, make sure you can import these modules on a Python shell or in a Python script without errors. For more information, see Set Up Python API for Polyspace.

  1. Import the required modules:

    import polyspace.project
import polyspace.test
import os

  2. Add source files and xUnit test files to a project. This example uses some example source and test files available with a Polyspace Test installation. Instead, you can use your own sources and tests.

    examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                            "examples", "pstest", "Getting_Started_Example")
polyspaceProject = polyspace.project.Project("newProject")
polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))
polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

  3. Set the coverage metric level to MCDC in the active test configuration of the project:

    coverageMetricLevel = polyspace.project.CoverageMetricLevel.MCDC
polyspaceProject.ActiveTestConfiguration.CoverageOptions.Level = coverageMetricLevel
    The other possible values instead of MCDC are STATEMENT, DECISION, CONDITION_DECISION, or NONE. For an explanation of the coverage metric levels, see Coverage metrics (-cov-metric-level).

  4. Run the tests added to the project with code coverage computation enabled.

    res = polyspace.test.run(
      polyspaceProject,
      ProfilingSelection=polyspace.test.ProfilingSelection.COVERAGE
)

  5. Extract the coverage results and view which coverage metric passes a specific threshold, for example, 50%.

    # Read coverage results
profilingResults = res.Profiling
coverageResults = profilingResults.Coverage

coverage_metrics = ["decision","condition","mcdc","statement","function call", \
"function", "function exit"]

# Show pass/fail status of coverage results based on coverage percentage
print("Coverage Threshold is 50%")
for metric in coverage_metrics:
  metric_details = coverageResults.getCoverageInfo(metric, "utils.c")
  if metric_details.TotalCount:
    pct = (metric_details.CoveredCount + metric_details.JustifiedCount) / metric_details.TotalCount
    print(f"{metric} coverage is {pct:.2%}\n ", " (FAIL)" if pct < 0.5 else " (PASS)")
  else:
    print(f"{metric} coverage not calculated\n")

    If you use the example source and test files, you see an output like this:

    decision coverage is 50.00%
   (PASS)
condition coverage is 40.00%
   (FAIL)
mcdc coverage is 10.00%
   (FAIL)
statement coverage is 57.14%
   (PASS)
function call coverage is 100%
   (PASS)
function coverage is 66.67%
   (PASS)
function exit coverage is 42.86%
   (FAIL)

In the previous example, in the source file utils.c, the function isGreaterThanSpeedLimit contains an unreachable branch because the if condition in this function is always true. In this example, you apply a filter to justify all missing code coverage results in this function.

  1. Get coverage results using the steps in the previous example:

    import polyspace.project
import polyspace.test
import os

# Add sources and tests
examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                            "examples", "pstest", "Getting_Started_Example")
polyspaceProject = polyspace.project.Project("newProject")
polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))
polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

# Configure coverage computation
coverageMetricLevel = polyspace.project.CoverageMetricLevel.MCDC
polyspaceProject.ActiveTestConfiguration.CoverageOptions.Level = coverageMetricLevel

# Run tests with coverage enabled
res = polyspace.test.run(
      polyspaceProject,
      ProfilingSelection=polyspace.test.ProfilingSelection.COVERAGE
)

# Read coverage results
profilingResults = res.Profiling
coverageResults = profilingResults.Coverage

  2. Inspect the decision coverage results for the isGreaterThanSpeedLimit function.

    result_isGreaterThanSpeedLimit = coverageResults.getCoverageInfo("decision", "utils.c","isGreaterThanSpeedLimit")
print(result_isGreaterThanSpeedLimit)

    Only one of the two decision outcomes of the if statement is covered by the tests.

    {
    TotalCount: 2
    CoveredCount: 1
    JustifiedCount: 0
    Details: [<DecisionCoverageDetail>]
}

  3. Define a filter to justify all missing code coverage results in the isGreaterThanSpeedLimit function. Apply this filter to the coverageResults object to obtain a filtered object filteredCoverageResults.

    filters = polyspace.test.CoverageFilters()
filters.Rules.createForFunction("isGreaterThanSpeedLimit", "utils.c", Rationale="Unreachable code", Status="justified")
filteredCoverageResults = coverageResults.applyFilters(filters)

    Instead of using the createForFunction method, you can also use:

    • The createForFile method to create a rule that applies to an entire file

    • The create method to create a rule that filters a specific decision or a specific decision outcome

    For more information on these methods, see the description of the Rules property of the polyspace.test.CoverageFilters class.

  4. Inspect the filtered decision coverage results for the isGreaterThanSpeedLimit function.

    filteredResult_isGreaterThanSpeedLimit = filteredCoverageResults.getCoverageInfo("decision", "utils.c","isGreaterThanSpeedLimit")
print(filteredResult_isGreaterThanSpeedLimit)

    The previously uncovered decision outcome has now been marked as justified.

    {
    TotalCount: 2
    CoveredCount: 1
    JustifiedCount: 1
    Details: [<DecisionCoverageDetail>]
}

Version History

Introduced in R2024b

expand all

See Also

| | | | | |

Topics