polyspace.test.AssessmentResult Class

Namespace: polyspace.test

(Python) Review results for a specific assessment in a test

Since R2024b

Description

This Python^® class contains test results for a specific assessment in a C/C++ test.

Creation

Syntax

assessmentResult = testCaseResult.AssessmentResults[assessmentIndex]

assessmentResult = testCaseResult.PassedAssessmentResults[assessmentIndex]

assessmentResult = testCaseResult.FailedAssessmentResults[assessmentIndex]

Description

assessmentResult = testCaseResult.AssessmentResults[assessmentIndex] loads the results of assessment number assessmentIndex:

• From a polyspace.test.TestCaseResult object testCaseResult.

• To a polyspace.test.AssessmentResult object assessmentResult.

Here, assessmentIndex is the assessment index starting from 0.

assessmentResult = testCaseResult.PassedAssessmentResults[assessmentIndex] reads the results of passing assessment number assessmentIndex:

• From a polyspace.test.TestCaseResult object testCaseResult.

• To a polyspace.test.AssessmentResult object assessmentResult.

Here, assessmentIndex is the assessment index starting from 0.

assessmentResult = testCaseResult.FailedAssessmentResults[assessmentIndex] reads the results of failing assessment number assessmentIndex:

• From a polyspace.test.TestCaseResult object testCaseResult.

• To a polyspace.test.AssessmentResult object assessmentResult.

Here, assessmentIndex is the assessment index starting from 0.

Properties

Id — Unique identifier for assessment result

string

Unique identifier for assessment result, for instance, "ea7222b5-e4fd-453f-ba3b-4c2a7019f9f8".

File — File containing assessment

string

File containing assessment.

Line — Line number of assessment

integer

Line number of assessment.

Expression — Expression evaluated by assessment

string

Expression evaluated by assessment, for instance, "lhs == rhs".

Type — Type of assessment

AssessmentType.NONE | AssessmentType.VERIFY | AssessmentType.ASSERT | AssessmentType.ASSUME

Type of assessment specified as an enum class object. The values are:

• AssessmentType.NONE

• AssessmentType.VERIFY

• AssessmentType.ASSERT

• AssessmentType.ASSUME

Passed — Whether assessment passed or failed

boolean

Status of assessment, specified as True if the assessment passes or False if the assessment fails.

Diagnostic — Details of assessment failure

string

Expected and actual values when assessment fails, for instance, "lhs="0" rhs="27"".

Msg — Message when failure occurs

string

Custom message when a test failure occurs. You can specify custom messages only for xUnit tests.

Phase — Test phase where assessment occurs

string

Test phase where assessment occurs, specified as one of the following:

• TestPhase.NONE

• TestPhase.SUITE

• TestPhase.SUITE_CONFIG

• TestPhase.SUITE_SETUP

• TestPhase.SUITE_TEARDOWN

• TestPhase.SUITE_TEST_SETUP

• TestPhase.SUITE_TEST_TEARDOWN

• TestPhase.TEST

• TestPhase.TEST_CONFIG

• TestPhase.TEST_SETUP

• TestPhase.TEST_TEARDOWN

Examples

Print File Names and Line Numbers of Failing Assessments

This example shows to print the names of failing test cases after running tests in a project.

In general, you generate and manage Polyspace^® Test™ results by using classes from the polyspace.project and polyspace.test modules. Before starting, make sure you can import these modules on a Python shell or in a Python script without errors. For more information, see Set Up Python API for Polyspace.

1. Import the required modules:

   import polyspace.project
   import polyspace.test
   import os

2. Add source files and xUnit test files to a project. This example uses some example source and test files available with a Polyspace Test installation. Instead, you can use your own sources and tests.

   examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                               "examples", "pstest", "Getting_Started_Example")
   polyspaceProject = polyspace.project.Project("newProject")
   polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
   polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))
   polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

3. Run the tests added to the project.

   res = polyspace.test.run(polyspaceProject)

4. Print the names of failing test cases (along with their suite names) and the file and line numbers of failing assessments in those tests.

   for suiteIndex in range(len(res.TestSuiteResults)):
       # Read result of test suite
       resTestSuite = res.TestSuiteResults[suiteIndex]
       if(resTestSuite.Status.Failed):
           for testIndex in range(len(resTestSuite.TestCaseResults)):
               # Read result of test case
               resTestCase = resTestSuite.TestCaseResults[testIndex]
               if(resTestCase.Status.Failed):
                   print(resTestSuite.Name + "/" + resTestCase.Name + "\n")
                   for assessmentIndex in range(len(resTestCase.FailedAssessmentResults)):
                       # Read result of failing assessment
                       resAssessment = resTestCase.FailedAssessmentResults[assessmentIndex]
                       print(resAssessment.File + ": Line " + str(resAssessment.Line))

   Alternatively, you can iterate over the test suites, cases and assessments as follows:

   for resTestSuite in res.TestSuiteResults:
       if resTestSuite.Status.Failed:
           for resTestCase in resTestSuite.TestCaseResults:
               if resTestCase.Status.Failed:
                   print(f"{resTestSuite.Name}/{resTestCase.Name}\n")
                   for resAssessment in resTestCase.FailedAssessmentResults:
                       print(f"{resAssessment.File}: Line {resAssessment.Line}")

Version History

Introduced in R2024b