|
|
|
| R2012a Documentation → Polyspace | |
Learn more about Polyspace |
|
| Contents | Index |
Polyspace verification supports two goals at the same time:
Reducing the cost of testing and validation
Improving software quality
You can use Polyspace verification in different ways depending on your development context and quality model.
This section provides sample workflows that show how to use Polyspace verification in a variety of development contexts.
This workflow applies to software developers and test groups using a standard development process. Before implementing Polyspace verification, these users fit the following criteria:
In Ada, no unit test tools or coverage tools are used – functional tests are performed just after coding.
In C, either no coding rules are used, or rules are not followed consistently.
The main goal of Polyspace verification is to improve productivity while maintaining or improving software quality. Verification helps developers and testers find and fix bugs more quickly than other processes. It also improves software quality by identifying bugs that otherwise might remain in the software.
In this process, the goal is not to completely prove the absence of errors. The goal is to deliver code of equal or better quality that other processes, while optimizing productivity to ensure a predictable time frame with minimal delays and costs.
This process involves file-by-file verification immediately after coding, and again just before functional testing.
The verification workflow consists of the following steps:
The project leader configures a Polyspace project to perform robustness verification, using default Polyspace options.
Each developer performs file-by-file verification as they write code, and reviews verification results.
The developer fixes all red errors and examines gray code identified by the verification.
The developer repeats steps 2 and 3 as needed, while completing the code.
Once a developer considers a file complete, they perform a final verification.
The developer fixes any red errors, examines gray code, and performs a selective orange review.
Using this approach, it is possible that some bugs may remain in unchecked oranges. However, the verification process represents a significant improvement from other testing methods.
When using verification to detect bugs:
Red and gray checks – Reviewing red and gray checks provides a quick method to identify real run-time errors in the code.
Orange checks – Selective orange review provides a method to identify potential run-time errors as quickly as possible. The time required to find one bug varies from 5 minutes to 1 hour, and is typically around 30 minutes. This represents an average of two minutes per orange check review, and a total of 20 orange checks per package in Ada and 60 orange checks per file in C.
Disadvantages to this approach:
Number of orange checks – If you do not use coding rules, your verification results will contain more orange checks.
Unreviewed orange checks – Some bugs may remain in unchecked oranges.
This workflow applies to software developers and test engineers working within development groups. These users are often developing software for embedded systems, and typically use coding rules.
These users typically want to find bugs early in the development cycle using a tool that is fast and iterative.
The goal of Polyspace verification is to improve software quality with equal or increased productivity.
Verification can prove the absence of run-time errors, while helping developers and testers find and fix any bugs more quickly than other processes.
This process involves both code analysis and code verification during the coding phase, and thorough review of verification results before module testing. It may also involve integration analysis before integration testing.
Workflow for Code Verification
The verification workflow consists of the following steps:
The project leader configures a Polyspace project to perform contextual verification. This involves:
Using Data Range Specifications (DRS) to define initialization ranges for input data. For example, if a variable "x" is read by functions in the file, and if x can be initialized to any value between 1 and 10, this information should be included in the DRS file.
Creates a "main" program to model call sequence, instead of using the automatically generated main.
Sets options to check the properties of some output variables. For example, if a variable "y" is returned by a function in the file and should always be returned with a value in the range 1 to 100, then Polyspace can flag instances where that range of values might be breached.
The project leader configures the project to check appropriate coding rules.
Each developer performs file-by-file verification as they write code, and reviews both coding rule violations and verification results.
The developer fixes any coding rule violations, fixes all red errors, examines gray code, and performs a selective orange review.
The developer repeats steps 2 and 3 as needed, while completing the code.
Once a developer considers a file complete, they perform a final verification.
The developer or tester performs an exhaustive orange review on the remaining orange checks.
Note The goal of the exhaustive orange review is to examine all orange checks that were not reviewed as part of previous reviews. This is possible when using coding rules because the total number of orange checks is reduced, and the remaining orange checks are likely to reveal problems with the code. |
Optionally, an additional verification can be performed during the integration phase. The purpose of this additional verification is to track integration bugs, and review:
Red and gray integration checks;
The remaining orange checks with a selective review: Integration bug tracking.
With this approach, Polyspace verification typically provides the following benefits:
Fewer orange checks in the verification results (improved selectivity). The number of orange checks is typically reduced to 3–5 per file, yielding an average of 1 bug. Often, several of the orange checks represent the same bug.
Fewer gray checks in the verification results.
Typically, each file requires two verifications before it can be checked-in to the configuration management system.
The average verification time is about 15 minutes.
If performing the optional verification to find integration bugs, you may see the following results. On a typical 50,000 line project:
A selective orange review may reveal one integration bug per hour of code review.
Selective orange review takes about 6 hours to complete. This is long enough to review orange checks throughout the whole application. This represents a step towards an exhaustive orange check review. However, spending more time is unlikely to be efficient, and will not guarantee that no bugs remain.
An exhaustive orange review would take between 4 and 6 days, assuming that 50,000 lines of code contains approximately 400–800 orange checks. Exhaustive orange review is typically recommended only for high-integrity code, where the consequences of a potential error justify the cost of the review.
This workflow applies to quality engineers who work outside of software development groups, and are responsible for independent verification of software quality and adherence to standards.
These users generally receive code late in the development cycle, and may even be verifying code that is written by outside suppliers or other external companies. They are concerned with not just detecting bugs, but measuring quality over time, and developing processes to measure, control, and improve product quality going forward.
The main goal of Polyspace verification is to control and evaluate the safety of an application.
The criteria used to evaluate code can vary widely depending on the criticality of the application, from no red errors to exhaustive oranges review. Typically, these criteria become increasingly stringent as a project advances from early, to intermediate, and eventually to final delivery.
For more information on defining these criteria, see Defining Software Quality Levels.
This process usually involves both code analysis and code verification before validation phase, and thorough review of verification results based on defined quality goals.
Note Verification is often performed multiple times, as multiple versions of the software are delivered. |
The verification workflow consists of the following steps:
Quality engineering group defines clear quality goals for the code to be written, including specific quality levels for each version of the code to be delivered (first, intermediate, or final delivery) For more information, see Defining Quality Goals.
Development group writes code according to established standards.
Development group delivers software to the quality engineering group.
The project leader configures the Polyspace project to meet the defined quality goals, as described in Defining a Verification Process to Meet Your Goals.
Quality engineers perform verification on the code.
Quality engineers review all red errors, gray code, and the number of orange checks defined in the process.
Note The number of orange checks reviewed often depends on the version of software being tested (first, intermediate, or final delivery). This can be defined by quality level (see Defining Software Quality Levels). |
Quality engineers create reports documenting the results of the verification, and communicate those results to the supplier.
Quality engineers repeat steps 5–7 for each version of the code delivered.
The benefits of code verification at this stage are the same as with other verification processes, but the cost of correcting faults is higher, because verification takes place late in the development cycle.
It is possible to perform an exhaustive orange review at this stage, but the cost of doing so can be high. If you want to review all orange checks at this phase, it is important to use development and verification processes that minimize the number of orange checks. This includes:
Developing code using strict coding and data rules.
Providing accurate manual stubs for all unresolved function calls.
Using DRS to provide accurate data ranges for all input variables.
Taking these steps will minimize the number of orange checks reported by the verification, and make it likely that any remaining orange checks represent true issues with the software.
This workflow applies to quality engineers who work with applications requiring outside quality certification, such as IEC 61508 certification or DO-178B qualification.
These users must perform a set of activities to meet certification requirements.
For information on using Polyspace products within an IEC
61508 certification environment, see the IEC Certification
Kit: Verification of C and C++ Code Using Polyspace Products.
For information on using Polyspace products within an DO-178B qualification environment, see the DO Qualification Kit: Polyspace Client/Server for C/C++ Tool Qualification Plan.
This workflow applies to users who have adopted model-based design to generate code for embedded application software.
These users generally use Polyspace software in combination with several other MathWorks® products, including Simulink®, Embedded Coder™ , and Simulink Design Verifier™ products. In many cases, these customers combine application components that are hand-written code with those created using generated code.
The goal of Polyspace verification is to improve the quality of the software by identifying implementation issues in the code, and ensuring the code is both semantically and logically correct.
Polyspace verification allows you to find run-time errors:
In hand-coded portions within the generated code
In the model used for production code generation
In the integration of hand-written and generated code
The workflow is different for hand-written code, generated code, and mixed code. Polyspace products can perform code verification as part of any of these workflows. The following figure shows a suggested verification workflow for hand-written and mixed code.
Workflow for Verification of Generated and Mixed Code
The verification workflow consists of the following steps:
The project leader configures a Polyspace project to meet defined quality goals.
Developers write hand-coded sections of the application.
Developers or testers perform Polyspace verification on any hand-coded sections within the generated code, and review verification results according to the established quality goals.
Developers create Simulink model based on requirements.
Developers validate model to ensure it is logically correct (using tools such as Simulink Model Advisor, and the Simulink Verification and Validation™ and Simulink Design Verifier products).
Developers generate code from the model.
Developers or testers perform Polyspace verification on the entire software component, including both hand-written and generated code.
Developers or testers review verification results according to the established quality goals.
Simulink Design Verifier verification can identify errors in textual designs or executable models that are not identified by other methods. The following table shows how errors in textual designs or executable models can appear in the resulting code.
Examples of Common Run-Time Errors
| Type of Error | Design or Model Errors | Code Errors |
|---|---|---|
| Arithmetic errors |
|
|
| Memory corruption |
|
|
Data truncation |
|
|
| Logic errors |
|
|
This workflow applies to project managers responsible for establishing check-in criteria for code at different development stages.
The goal of Polyspace verification is to test that code meets established quality criteria before being checked in at each development stage.
The verification workflow consists of the following steps:
Project manager defines quality goals, including individual quality levels for each stage of the development cycle.
Project leader configures a Polyspace project to meet quality goals.
Developers or testers run verification at the following stages:
Daily check-in — On the files currently under development. Compilation must complete without the permissive option.
Pre-unit test check-in — On the files currently under development.
Pre-integration test check-in — On the whole project, ensuring that compilation can complete without the permissive option. This stage differs from daily check-in because link errors are highlighted.
Pre-build for integration test check-in — On the whole project, with all multitasking aspects accounted for as appropriate.
Pre-peer review check-in — On the whole project, with all multitasking aspects accounted for as appropriate.
Developers or testers review verification results for each check-in activity to ensure the code meets the appropriate quality level. For example, the transition criterion could be: "No bug found within 20 minutes of selective orange review"
 | Implementing a Process for Polyspace Verification | Setting Up a Verification Project |  |
| © 1984-2012- The MathWorks, Inc. - Site Help - Patents - Trademarks - Privacy Policy - Preventing Piracy - RSS |