Documentation

Customizing Software Quality Objectives

About Customizing Software Quality Objectives

When you run your first verification to produce metrics, Polyspace® software uses predefined software quality objectives (SQO) to evaluate quality. In addition, when you use Polyspace Metrics for the first time, Polyspace creates the following XML file that contains definitions of these software quality objectives:

RemoteDataFolder/Custom-SQO-Definitions.xml

RemoteDataFolder is the folder where Polyspace stores data generated by remote verifications. See Modify Polyspace Server Configuration in the Polyspace Installation Guide.

If you want to customize SQOs and modify the way quality is evaluated, you must change Custom-SQO-Definitions.xml. This XML file has the following form:

<?xml version="1.0" encoding="utf-8"?>
<MetricsDefinitions>
        SQO Level 2
        SQO Level 3
        SQO Level 4
        SQO Level 5
        SQO Level 6
        SQO Exhaustive
        Run-Time Checks Set 1
        Run-Time Checks Set 2
        Run-Time Checks Set 3
        Status Acronym 1
        Status Acronym 2
</MetricsDefinitions>

The following topics provide information about MetricsDefinitions elements and how SQO levels are calculated. Use this information when you modify or create elements.

SQO Level 2

The default SQO Level 2 element is:

<SQO ID="SQO-2" ParentID="SQO-1">
        <Num_Unjustified_Red>0</Num_Unjustified_Red>
        <Num_Unjustified_NT_Constructs>0</Num_Unjustified_NT_Constructs>
</SQO>

To fulfill requirements of SQO Level 2, the code must meet the requirements of SQO Level 1 and the following:

  • Number of unjustified red checks Num_Unjustified_Red must not be greater than the threshold (default is zero)

  • Number of unjustified NTC and NTL checks Num_Unjustified_NT_Constructs must not be greater than the threshold (default is zero)

SQO Level 3

The default SQO Level 3 element is:

<SQO ID="SQO-3" ParentID="SQO-2">
        <Num_Unjustified_Gray>0</Num_Unjustified_Gray>
</SQO>

To fulfill requirements of SQO Level 3, the code must meet the requirements of SQO Level 2 and the number of unjustified UNR checks must not exceed the threshold (default is zero).

SQO Level 4

The default SQO Level 4 element is:

<SQO ID="SQO-4" ParentID="SQO-3">
        <Percentage_Proven_Or_Justified>
            Runtime_Checks_Set_1
        </Percentage_Proven_Or_Justified>
</SQO>

To fulfill requirements of SQO Level 4, the code must meet the requirements of SQO Level 3 and the following ratio as a percentage

(green checks + justified orange checks) / (green checks + all orange checks)

must not be less than the thresholds specified by Run-Time Checks Set 1.

SQO Level 5

The default SQO Level 5 element is:

<SQO ID="SQO-5" ParentID="SQO-4">
        <Percentage_Proven_Or_Justified>
            Runtime_Checks_Set_2
        </Percentage_Proven_Or_Justified>
</SQO>

To fulfill requirements of SQO Level 5, the code must meet the requirements of SQO Level 4 and the percentage of green and justified checks must not be less than the thresholds specified by Run-Time Checks Set 2.

SQO Level 6

The default SQO Level 6 element is:

<SQO ID="SQO-6" ParentID="SQO-5">
        <Percentage_Proven_Or_Justified>
            Runtime_Checks_Set_3
        </Percentage_Proven_Or_Justified>
</SQO>

To fulfill requirements of SQO Level 6, the code must meet the requirements of SQO Level 5 and the percentage of green and justified checks must not be less than the thresholds specified by Run-Time Checks Set 3.

SQO Exhaustive

The default Exhaustive element is:

<SQO ID="Exhaustive" ParentID="SQO-1">
        <Num_Unjustified_Red>0</Num_Unjustified_Red>
        <Num_Unjustified_NT_Constructs>0</Num_Unjustified_NT_Constructs>
        <Num_Unjustified_Gray>0</Num_Unjustified_Gray>
        <Percentage_Proven_Or_Justified>100</Percentage_Proven_Or_Justified>
</SQO>

Run-Time Checks Set 1

The Run-Time Checks Set 1 is composed of Check elements with data that specify thresholds. The Name and Type attribute in each Check element defines a run-time check, while the element data specifies a threshold in percentage. The default structure of Run-Time Checks Set 1 is:

<RuntimeChecksSet ID="Runtime_Checks_Set_1">
      <Check Name="OBAI">80</Check>
      <Check Name="ZDV" Type="Scalar">80</Check>
      <Check Name="ZDV" Type="Float">80</Check>
      <Check Name="NIVL">80</Check>
      <Check Name="NIV">60</Check>
      <Check Name="IRV">80</Check>
      <Check Name="FRIV">80</Check>
      <Check Name="FRV">80</Check>
      <Check Name="OVFL" Type="Scalar">60</Check>
      <Check Name="OVFL" Type="Float">60</Check>
      <Check Name="IDP">60</Check>
      <Check Name="NIP">60</Check>
      <Check Name="POW">80</Check>
      <Check Name="SHF">80</Check>
      <Check Name="COR">60</Check>
      <Check Name="NNR">50</Check>
      <Check Name="EXCP">50</Check>
      <Check Name="EXC">50</Check>
      <Check Name="NNT">50</Check>
      <Check Name="CPP">50</Check>
      <Check Name="OOP">50</Check>
      <Check Name="ASRT">60</Check>
</RuntimeChecksSet>

When you use Run-Time Checks Set 1 in evaluating code quality, the software calculates the following ratio as a percentage for each run-time check in the set:

(green checks + justified orange checks)/(green checks + all orange checks)

If the percentage values do not exceed the thresholds in the set, the code meets the quality level.

To modify the default set, you can change the check threshold values.

Run-Time Checks Set 2

This set is similar to Run-Time Checks Set 1, but has more stringent threshold values.

 <RuntimeChecksSet ID="Runtime_Checks_Set_2">
      <Check Name="OBAI">90</Check>
      <Check Name="ZDV" Type="Scalar">90</Check>
      <Check Name="ZDV" Type="Float">90</Check>
      <Check Name="NIVL">90</Check>
      <Check Name="NIV">70</Check>
      <Check Name="IRV">90</Check>
      <Check Name="FRIV">90</Check>
      <Check Name="FRV">90</Check>
      <Check Name="OVFL" Type="Scalar">80</Check>
      <Check Name="OVFL" Type="Float">80</Check>
      <Check Name="IDP">70</Check>
      <Check Name="NIP">70</Check>
      <Check Name="POW">90</Check>
      <Check Name="SHF">90</Check>
      <Check Name="COR">80</Check>
      <Check Name="NNR">70</Check>
      <Check Name="EXCP">70</Check>
      <Check Name="EXC">70</Check>
      <Check Name="NNT">70</Check>
      <Check Name="CPP">70</Check>
      <Check Name="OOP">70</Check>
      <Check Name="ASRT">80</Check>
</RuntimeChecksSet>

Run-Time Checks Set 3

This set is similar to Run-Time Checks Set 1, but has more stringent threshold values.

<RuntimeChecksSet ID="Runtime_Checks_Set_3">
      <Check Name="OBAI">100</Check>
      <Check Name="ZDV" Type="Scalar">100</Check>
      <Check Name="ZDV" Type="Float">100</Check>
      <Check Name="NIVL">100</Check>
      <Check Name="NIV">80</Check>
      <Check Name="IRV">100</Check>
      <Check Name="FRIV">100</Check>
      <Check Name="FRV">100</Check>
      <Check Name="OVFL" Type="Scalar">100</Check>
      <Check Name="OVFL" Type="Float">100</Check>
      <Check Name="IDP">80</Check>
      <Check Name="NIP">80</Check>
      <Check Name="POW">100</Check>
      <Check Name="SHF">100</Check>
      <Check Name="COR">100</Check>
      <Check Name="NNR">90</Check>
      <Check Name="EXCP">90</Check>
      <Check Name="EXC">90</Check>
      <Check Name="NNT">90</Check>
      <Check Name="CPP">90</Check>
      <Check Name="OOP">90</Check>
      <Check Name="ASRT">100</Check>
   </RuntimeChecksSet>

Status Acronyms

When you click a link, StatusAcronym elements are passed to the Polyspace verification environment. This feature allows you to define, through your Polyspace server, additional items for the drop-down list of the Status field in Result Details. See Review Run-Time Checks.

Polyspace Metrics provides the following default elements:

<StatusAcronym Justified="yes" Name="Justify with code/model annotations"/>
<StatusAcronym Justified="yes" Name="No action planned"/>

The Name attribute specifies the name that appears on the Status field drop-down list. If you specify the Justify attribute to yes, then when you select the item, for example, No action planned, the software automatically selects the Justified check box. If you do not specify the Justify attribute, then the Justified check box is not selected automatically.

You can remove the default elements and create new StatusAcronym elements, which are available to all users of your Polyspace server.

Was this topic helpful?