MATLAB Examples

Test a Model for Continuous Integration Systems

This example shows how to use MATLAB® Unit Test to test a model, and use the TAPPlugin to create TAP results. You can use TAP with CI systems. The model is a controller-plant system of a flight controller, aircraft model, and environment model.

Create a test suite and a test runner, and customize the runner with the plugin that creates the TAP file. When you run the test, it fails on several iterations. The results are written to the TAP file.

Before performing this example, set the working directory to a writable location on the path.

1. Open the Model

open_system(fullfile(matlabroot,'examples','simulinktest',...
    'sltestF14ParameterSweep.slx'))

2. Open the Test File

The test case creates a square wave input to the controller, and sweeps through 25 iterations of the parameters a and b. It compares the alpha output to a baseline with a tolerance of 0.0046 and fails any output which exceeds this tolerance.

sltest.testmanager.view;
sltest.testmanager.load(fullfile(matlabroot,'examples','simulinktest',...
    'f14ParameterSweepTest.mldatx'));

3. Import the TestRunner, TestSuite, TAPPlugin, and ToFile classes.

import matlab.unittest.TestRunner
import matlab.unittest.TestSuite
import matlab.unittest.plugins.TAPPlugin
import matlab.unittest.plugins.ToFile

4. Create the test suite object.

suite = testsuite(fullfile(matlabroot,'examples','simulinktest',...
    'f14ParameterSweepTest.mldatx'))
suite = 

  1x25 Test array with properties:

    Name
    BaseFolder
    ProcedureName
    SharedTestFixtures
    Parameterization
    Tags

Tests Include:
   25 Unique Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

5. Create the test runner object, and set it to display output to the command window.

f14runner = TestRunner.withTextOutput;

6. Create a TAP plugin that sends output to the file F14TapOutput.tap.

tapFile = 'F14TapOutput.tap';
plugin = TAPPlugin.producingVersion13(ToFile(tapFile));

7. Add the plugin to the test runner.

addPlugin(f14runner,plugin)

8. Run the test. The test fails several iterations in which the delta between the signal output and the baseline exceeds the tolerance.

result = run(f14runner,suite);
Running f14ParameterSweepTest > New Test Suite 1
..........
........
================================================================================
Verification failed in f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration19).

    ---------------------
    Framework Diagnostic:
    ---------------------
    Failed criteria: Baseline
    --> Logs:
            Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
    --> Simulink Test Manager Results:
            Results: 2017-Jul-31 21:17:47/Iterations Parameter Sweep/Scripted_Iteration19
================================================================================
..
...
================================================================================
Verification failed in f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration24).

    ---------------------
    Framework Diagnostic:
    ---------------------
    Failed criteria: Baseline
    --> Logs:
            Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
    --> Simulink Test Manager Results:
            Results: 2017-Jul-31 21:17:53/Iterations Parameter Sweep/Scripted_Iteration24
================================================================================
.
================================================================================
Verification failed in f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration25).

    ---------------------
    Framework Diagnostic:
    ---------------------
    Failed criteria: Baseline
    --> Logs:
            Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
    --> Simulink Test Manager Results:
            Results: 2017-Jul-31 21:17:55/Iterations Parameter Sweep/Scripted_Iteration25
================================================================================
.
Done f14ParameterSweepTest > New Test Suite 1
__________

Failure Summary:

     Name                                                                                                         Failed  Incomplete  Reason(s)
    ==========================================================================================================================================================
     f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration19)    X                 Failed by verification.
    ----------------------------------------------------------------------------------------------------------------------------------------------------------
     f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration24)    X                 Failed by verification.
    ----------------------------------------------------------------------------------------------------------------------------------------------------------
     f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration25)    X                 Failed by verification.
    

9. Display the results from the TAP file.

disp(fileread(tapFile))
TAP version 13
1..25
ok 1 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration1)
ok 2 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration2)
ok 3 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration3)
ok 4 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration4)
ok 5 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration5)
ok 6 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration6)
ok 7 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration7)
ok 8 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration8)
ok 9 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration9)
ok 10 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration10)
ok 11 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration11)
ok 12 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration12)
ok 13 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration13)
ok 14 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration14)
ok 15 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration15)
ok 16 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration16)
ok 17 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration17)
ok 18 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration18)
not ok 19 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration19)
    ---
    Event:
        Event Name: 'VerificationFailed'
        Event Location: 'f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration19)'
        Framework Diagnostic: |
            Failed criteria: Baseline
            --> Logs:
                    Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
            --> Simulink Test Manager Results:
                    Results: 2017-Jul-31 21:17:47/Iterations Parameter Sweep/Scripted_Iteration19
    ...
ok 20 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration20)
ok 21 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration21)
ok 22 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration22)
ok 23 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration23)
not ok 24 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration24)
    ---
    Event:
        Event Name: 'VerificationFailed'
        Event Location: 'f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration24)'
        Framework Diagnostic: |
            Failed criteria: Baseline
            --> Logs:
                    Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
            --> Simulink Test Manager Results:
                    Results: 2017-Jul-31 21:17:53/Iterations Parameter Sweep/Scripted_Iteration24
    ...
not ok 25 - f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration25)
    ---
    Event:
        Event Name: 'VerificationFailed'
        Event Location: 'f14ParameterSweepTest > New Test Suite 1/Iterations Parameter Sweep(ScriptedIteration=Scripted_Iteration25)'
        Framework Diagnostic: |
            Failed criteria: Baseline
            --> Logs:
                    Inputs may not be compatible for simulation. Test results might not be accurate. Click here for more information on external input mapping.
            --> Simulink Test Manager Results:
                    Results: 2017-Jul-31 21:17:55/Iterations Parameter Sweep/Scripted_Iteration25
    ...


sltest.testmanager.clearResults
sltest.testmanager.clear
sltest.testmanager.close
close_system('sltestF14ParameterSweep',0)