Exception Safety and Incremental Teardown for Unit Tests

2 views (last 30 days)
Under help topic "Write Setup and Teardown Code Using Classes", it is recommended that instead of using teardown method, "addTeardown" should be used because (based on the help content) "This guarantees the Teardown is executed in the reverse order of the setup and also ensures that the test content is exception safe".
What can go wrong in case a single teardown method is used?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 14 Dec 2018
There are two ways to define Teardown code for MATLAB Unit Tests:
  1. Creating a single "TestMethodTeardown" method
  2. Calling the "addTeardown" method on individual fixtures
The "addTeardown" method has certain advantages related to Exception Safety and Incremental Teardown. Consider the following two examples which illustrates this advantage. The first one creates a single "TestMethodTeardown" method and the second one uses the "addTeardown" method.
classdef MyTestClass < matlab.unittest.TestCase
properties
MyMatrix
MyFigureA
MyFigureB
end
methods (TestMethodSetup)
function setupMyTestClass(testCase)
try
% Creating MyFigureA
testCase.MyFigureA = figure(1);
testCase.MyFigureA.Name = "Figure A";
fprintf("Created %s.\n",testCase.MyFigureA.Name);
% Some error occurs here
testCase.MyMatrix = magic(2) * magic(3); % Invalid Matrix Multiplication
% Creating MyFigureB
testCase.MyFigureB = figure(2);
testCase.MyFigureB.Name = "Figure B";
fprintf("Created %s.\n",testCase.MyFigureB.Name);
catch mExp
error(mExp.message);
end
end
end
methods (TestMethodTeardown)
function teardownMyTestClass(testCase)
fprintf("Closing %s.\n",testCase.MyFigureA.Name);
close(testCase.MyFigureA);
fprintf("Closing %s.\n",testCase.MyFigureB.Name);
close(testCase.MyFigureB);
end
end
methods (Test)
function startMyTest(testCase)
fprintf("Running the Test.\n");
end
end
end
The code attempts to initialize a matrix and two figures before implementing the Unit Test and deletes the figures upon test completion. Test teardown is implemented by the method "teardownMyTestClass".
Although this is a straightforward method to define fixture teardown, certain problems may arise. If an exception is thrown in the middle of the "TestMethodSetup" method, the framework will attempt to execute the full content of the fixture teardown code. This may result in trying to teardown fixtures that were never setup, and unwanted errors could follow.
For this example, an error will occur when the program attempts to delete "MyFigureB" during teardown, because it does not exist. Also, if more than one "TestMethodSetup" and/or "TestMethodTeardown" methods are defined in the same class, then it is possible that the teardown does not occur in the opposite order of the setup.
To alleviate this issue, the "addTeardown" method can be utilized because it facilitates a deterministically ordered teardown process that is also exception safe. The order in which this teardown occurs is the opposite of the order the fixtures were added. This avoids executing teardown code for uninitialized class members.
classdef MyTestClass < matlab.unittest.TestCase
properties
MyMatrix
MyFigureA
MyFigureB
MyFigureC
end
methods (TestMethodSetup)
function setupMyTestClass(testCase)
try
% Creating MyFigureA
testCase.MyFigureA = figure(1);
testCase.MyFigureA.Name = "Figure A";
fprintf("Created %s.\n",testCase.MyFigureA.Name);
testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureA);
% Creating MyFigureB
testCase.MyFigureB = figure(2);
testCase.MyFigureB.Name = "Figure B";
fprintf("Created %s.\n",testCase.MyFigureB.Name);
testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureB);
% Some error occurs here
testCase.MyMatrix = magic(2) * magic(3); % Invalid Matrix Multiplication
% Creating MyFigureC
testCase.MyFigureC = figure(3);
testCase.MyFigureC.Name = "Figure C";
fprintf("Created %s.\n",testCase.MyFigureC.Name);
testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureC);
catch mExp
error(mExp.message);
end
end
end
methods
function closeTheFigure(testCase, fig)
fprintf("Closing %s.\n",fig.Name);
close(fig);
end
end
methods (Test)
function startMyTest(testCase)
fprintf("Running the Test.\n");
% Code to run the test goes here
end
end
end
Execution occurs in the following order:
1. "MyFigureA" gets created
2. "MyFigureB" gets created
3. An error is thrown because of the invalid matrix multiplication
4. Teardown method for "MyFigureB" is called
5. Teardown method for "MyFigureA" is called
Note that the program does not attempt to close "MyFigureC" in this case.

More Answers (1)

Andy Campbell
Andy Campbell on 17 Dec 2018
There is a blog post that describes the addTeardown method and some of its benefits over other methods of managing test and test fixture state. Here is the link:
Enjoy!

Products


Release

R2017a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!