This example shows how to use the Metrics Dashboard with open-source tools GitLab™ and Jenkins™ to test and refine your model in a continuous integration systems workflow. Continuous integration is the practice of merging all developer working copies of project files to a shared mainline. This workflow saves time and improves quality by maintaining version control and automating and standardizing testing.
This example refers to a project that contains the shipped project matlab:sldemo_slproject_airframe and these additional files which are relevant to this example:
A MATLAB script that specifies metric thresholds and customizes the Metrics Dashboard.
A MATLAB unit test that collects metric data and checks whether there are metric threshold violations.
The example uses the Jenkins continuous integration server to run the MATLAB unit test to determine if there are metric threshold violations. Jenkins archives test results for you to download and investigate locally. GitLab is an online Git repository manager that you can configure to work with Jenkins. This diagram shows how Simulink Check, GitLab, and Jenkins work together in a continuous integration workflow.
The project contains all model, data, and configuration files including these files which are required for this example:
A MATLAB unit test that collects metric data for the project and checks that the model files contain no metric threshold violations. For more information on the MATLAB Unit tests, see Script-Based Unit Tests (MATLAB).
setup.m file that activates the configuration XML files
that define metric thresholds, set custom metric families, and customizes the
Metrics Dashboard layout. For this example, this code is the
function setup % refresh Model Advisor customizations Advisor.Manager.refresh_customizations(); % set metric configuration with thresholds configFile = fullfile(pwd, 'config', 'MyConfiguration.xml'); slmetric.config.setActiveConfiguration(configFile); uiconf = fullfile(pwd, 'config', 'MyDashboardConfiguration.xml'); slmetric.dashboard.setActiveConfiguration(uiconf); end
For more information on how to customize the Metrics Dashboard, see Customize Metrics Dashboard Layout and Functionality.
sl_customization.m file that activates the Model Advisor
configuration file to customize the Model Advisor checks. For more information,
see Create and Add Custom Checks - Basic Examples
run script that executes during a Jenkins build. For this
example, this code is in the
% script executed during Jenkins build function run(IN_CI) if (IN_CI) jenkins_workspace = getenv('WORKSPACE'); cd(jenkins_workspace); end % open the sl project slproj = simulinkproject(pwd); % execute tests runUnitTest(); slproj.close(); if IN_CI exit end end
cleanup.m file that resets the active metric
configuration to the default configuration. For this example, this code is in
function cleanup rmpath(fullfile(pwd, 'data')); Advisor.Manager.refresh_customizations(); % reset active metric configuration to default slmetric.config.setActiveConfiguration(''); slmetric.dashboard.setActiveConfiguration(''); end
.gitignore file that verifies that derived artifacts are
not checked into GitLab. This code is in the
work/** reports/** *.asv *.autosave
Create a GitLab project for source-controlling your Project. For more information, see https://docs.gitlab.com/ee/README.html.
Install the Git Client.
Set up a branching workflow. With GitLab, from the main branch, create a temporary branch for implementing changes to the model files. Integration engineers can use Jenkins test results to decide whether to merge a temporary branch into the master branch. For more information, see
Under Settings > Repository, protect the master branch by enforcing the use of merge requests when developers want to merge their changes into the master branch.
Under Settings, on the Integrations page, add a webhook to the URL of your Jenkins project. This webhook triggers a build job on the Jenkins server.
Install GitLab and Tap plugins. The MATLAB unit test uses the TAPPlugin to stream
results to a
.tap file. To enable communication of test status from
MATLAB to the Jenkins job, Jenkins imports the
Create a Jenkins project. Specify these configurations:
In your Jenkins project, click Configure.
On the General tab, specify a project name.
On the Source Code Management tab, for the Repository URL field, specify the URL of your GitLab repository.
On the Build Triggers tab, select Build when a change is pushed to GitLab.
On the Build tab, execute MATLAB to call the run script. The run script opens the project and runs all unit tests. For the project in this example, the code is:
matlab -nodisplay -r... "cd /var/lib/jenkins/workspace/'18b Metrics CI Demo'; run(true)"
In the Post-build Actions tab, configure the TAP
plugin to publish TAP results to Jenkins. In the Test
Results field, specify
Files to archive, specify
The TAP plugin shows details from the MATLAB Unit test in the extended results of the job. The Jenkins archiving infrastructure saves derived artifacts that are generated during a Jenkins build.
After setting up your project, Jenkins, and GitLab, follow the continuous integration workflow.
Create a local clone of the GitLab repository. See Clone from Git Repository (MATLAB).
In Simulink, navigate to the local GitLab repository.
Make any necessary changes to the project files.
Simulate the model and validate the output in the Simulation Data Inspector.
Run MATLAB unit tests. For more information, see
In GitLab, create a merge request. Select the feature branch as source branch and the target branch as master. Click Compare Branches and Continue.
If the feature is not fully implemented, mark the merge request as a work in progress by adding the letters WIP: at the beginning of the request. If the merge request is not marked WIP:, it immediately triggers a build after creation.
Click Submit Merge Request.
If the letters WIP: are not at the beginning of the merge request, the push command triggers a Jenkins build. In the Jenkins Setup part of this example, you configured Jenkins to perform a build when you pushed changes to GitLab. To remove the letters, click Resolve WIP status.
Navigate to the Jenkins project. In Build History, you can see the build status.
Click the Build.
Click Tap Test Results.
For this example, the
test did not pass for three metrics because these metrics did not meet the
thresholds. To investigate this data, you must download the data
Download the archived results to a local Git repository workspace.
Unzip the downloaded files. Copy the
work/ folders to the respective folders in the local
To explore the results, open the project and the Metrics Dashboard.
To resolve the test failures, make the necessary updates to the models. Push the changes to the feature branch in GitLab.
Integration engineers can use Jenkins test results to decide when it is acceptable to perform the merge of the temporary branch into the master branch.