Main Content

Include Results from External Sources in Verification Status

Requirements Toolbox™ allows you to include the verification status of results from external sources in the Requirements Editor. You can summarize requirements verification status, author your custom domain registration, and write custom logic to fetch the results. For more information, see Review Requirements Verification Status.

You can also include test results from:

  • Continuous integration (CI) servers such as Jenkins

  • Custom results updated manually or with test scripts

You can create custom link type registrations that interpret test results from the external environment into language specific to your development environment. See, Custom Link Types.

You can use built-in verification services to interpret result files for most common cases, such as JUnit and TAP (Test Anything Protocol), to include external test results in the requirements verification status.

When you include the verification status of external test results in your requirements:

  • The external results are listed in the Verified column of the Requirements Editor, along with results from other sources, such as Model Verification blocks and Simulink® Test™ test files.

  • Pass/fail indication is reflected in requirement links.

  • Result status is automatically aggregated across requirement hierarchies.

  • Result status automatically updates as requirements are added or removed.

How to Populate Verification Results from External Sources

Commonly, external test results are run and managed outside of the MATLAB® environment. Test results can be the product of:

  • Running test scripts or other programs that generate a result file

  • Running a MATLAB Unit Test test case with a custom TestRunner object, with or without a CI server

You can create links to the test results by either:

  • Linking directly to a result file. The external result artifact is used as the link destination and the requirements are used as the links source. To create custom link type, you must know:

    1. The file location

    2. The file format (for example, JUnit or Excel®)

    For details, see Linking to a Result File.

  • Linking to a test script and providing code that fetches results based on that test location. The external test artifacts are used as the link destination and the requirements are used as the link source. Your custom logic in the GetResultFcn function should locate the result artifact that corresponds to the test artifact and fetch results from that result artifact. See Linking to a Test Script.

The following steps are used to create the links from external sources and populate verification statuses from them:

  1. Create a custom link type: In the Requirements Management Interface (RMI), create a custom link type for your test result file:

    1. Write a MATLAB function that implements the custom link type. The GetResultFcn is implemented in the custom link type. For more information, see Links and Link Types.

    2. Save the function on the MATLAB path.

    For details, see Custom Link Type Registration.

  2. Register the custom link type: See Custom Link Type Registration. After registration, the link type is available in the Outgoing Links Editor in the Document type menu.

  3. Link from the requirement to the test result file or test script: Use the Outgoing Links Editor or slreq.createLink to link from the requirements to the results file.

  4. Display the verification status: In the Requirements Editor, view the Verified column to view the verification status. For details, see Display Verification Status.

  5. Refresh the requirements view: After the tests run, refresh the verification status by clicking the Refresh button.

You can include the verification status from external sources in your requirements report by clicking Report > Generate Report from the Requirements Editor.

When populating verification results from external sources:

  • Test the GetResultFcn code before integrating the code with rmi register. For more information about GetResultFcn, see Links and Link Types.

  • Confirm the custom link type registration in the Outgoing Links Editor.

  • Use caching to improve the performance for cases where a single file contains a result for many links.

  • Insert break points into the GetResultFcn code and use the Refresh button to re-execute it.

  • When using Projects, register and unregister the custom link type when using in project startup or shutdown scripts.

Related Examples

More About