No BSD License  

Highlights from
MUnit: a unit testing framework in Matlab

MUnit: a unit testing framework in Matlab

by

 

05 Jun 2006 (Updated )

A complete unit testing framework in Matlab

A unit testing framework for Matlab

A unit testing framework for Matlab

In the Extreme Programming (ExP) techniques the Test Driven Development
(TDD) method is a fundamental technique to control the complexity of the
code evolution and to check the consistency of the system response after
refactoring or massive changes.
In other languages a set of tools allow to write tests without working
on them more than required, a well known example is JUnit, but C++Unit,
PHPUnit and a plethora of similar tools exists for a great number of
languages. For Matlab a set of proposals can be found in the Net, but I
think that none of them can give simplicity and power as the other
testing tools in the other languages. This is my personal proposal of a
powerful tool that allow to use test cases, test sets, fixtures,
automatic tests collecting, extendible and reimplementable test runners
(to have a GUI version, a text version, a personal version, etc.), and
all what a developer can wish in a testing framework. All that the user
have to do is to add the rows:
   % Desc of the testset.
   function out = SfactoryTemplate(in)
   % This code must be inserted in every test file.
   out = str2func(in);
at the beginning of a test file; write (optional) fixtures as
sub-functions with signatures:
   % This is the setup function, use this for fixtures:
   function tso = MUnitSetUp(ts,event)
   % Comment to the setup callback
   % Code that generates the new ts:
   tso = ts;
   % This is the shoutdown function, use this for fixtures:
   function tso = MUnitShoutDown(ts,event)
   % Comment to the shoutdown callback
   % Code that generates the new ts:
   tso = ts;
generating (optional) callbacks to the start and the end of a test using
the functions:
   % This is the reset callback called before each testunit test
   function tuo = MUnitReset(tu,event)
   % Comment to the reset callback
   % Code that prepare the tuo test:
   tuo = tu;
   % This is the reset callback called after each testunit test
   function tuo = MUnitDone(tu,event)
   % Comment to the done callback
   % Code that analize the tuo test:
   tuo = tu;
and generating tests functions using the signatures:
This is an example of function used as test:
   function [tuo,status,tdatao] = MUnitTest_ATest(tu,tdata)
   % Comment
   %  Code that computes the new version of tu and tdata, then returns
   % a boolena status.
   status = true;
   % A list of asserts:
   try
       % Check of a boolean assertion:
       assertTrue(tu,<booleanExpression>,'Comment');
       % Check an operation that can fail with error:
       try <operation>
       catch failure(tu,'Comment'); end
   catch
       % Bad!!!
       status = false;
   end
where the discriminative part is the "MUnitTest_" prefix.
And another useful task can be the organization of the testing files in
a directory structure.

All this is, in my opinion, what a developer can ask to a unit testing framework. So, try it.. and have a funny testing :)

Contents

Generation of a test set

Here a tree of tests is automatically collected parsing a tests directory
of test files. The "testcollect" function recursively iterate in a
directory and search for m-files: for each m-file automatically detect
test functions and fixtures and generates the testsets and testcases.
A test set can be generated also giving a cell of filenames or dirnames
(mixed) to allow multiple sparse tests to be added, but the more
comfortable way to generate tests is to organize them in a directory. The
order can be controlled using the alphabetical ordering of the filenames
(for example using numbers as prefix of filenames, like "00-TestBlaBla.m"
and "01-TestOtherBlaBla.m").
% Collecting tests:
tests = testcollect('tests');

Preparing for the tests

The test tree must be prepared with the callbacks of the runner to
interact with it, so this step must be done.
% Selecting the runner:
runner = @testrunnerTextSimple;

% Adding the runner callbacks:
tests = feval(runner,tests,true);

Executing tests

Here we wish to execute the tests using the runner, this can be done
setting to false the third parameter (or omitting it). Here the tests are
executed and an error reporting log is written in the command window.
Here a set of tests fails (I write them to obtain this result) and a set
of actions are executed to show that the callback technique allow to hook
the tests and generate additional work.
% Running:
testsResults = feval(runner,tests);
==> Testing process starts at 05-Jun-2006 16:51:23
tests
SfactoryTemplate

Some tests FAILS in 0.003 seconds!
factoryTests
StestTestSet01
.F
***     Cos: FAILURE! "Check the function 'cos'"
          	lasterr: Cos(pi/2) isn't 0!

Some tests FAILS in 0.003 seconds!
anotherTestSet
StestTestSet02
Setting up...
Warning: Matrix is close to singular or badly scaled.
         Results may be inaccurate. RCOND = 1.541976e-18.
F
***    Inv1: FAILURE! "Check the function 'inv'"
          	lasterr: The inverse of the non-singular matrix m1 isn't really the inverse!
Shoutting down...
Setting up...
.Shoutting down...

Some tests FAILS in 0.002 seconds!
StestTestSet03
..
All tests SUCCESSES in 2.060 seconds!

Some tests FAILS in 2.061 seconds!

Some tests FAILS in 2.063 seconds!

Some tests FAILS in 2.064 seconds!
==> Testing process ends at 05-Jun-2006 16:51:27

All in a single call

All this can be done using a single call to the funciton "testRun"
giving as parameters the dirname of the tests and the test runner.
% All done here:
testsResults = testRun('tests',@testrunnerTextSimple);
==> Testing process starts at 05-Jun-2006 16:51:28
tests
SfactoryTemplate

Some tests FAILS in 0.002 seconds!
factoryTests
StestTestSet01
.F
***     Cos: FAILURE! "Check the function 'cos'"
          	lasterr: Cos(pi/2) isn't 0!

Some tests FAILS in 0.003 seconds!
anotherTestSet
StestTestSet02
Setting up...
Warning: Matrix is close to singular or badly scaled.
         Results may be inaccurate. RCOND = 1.541976e-18.
F
***    Inv1: FAILURE! "Check the function 'inv'"
          	lasterr: The inverse of the non-singular matrix m1 isn't really the inverse!
Shoutting down...
Setting up...
.Shoutting down...

Some tests FAILS in 0.002 seconds!
StestTestSet03
..
All tests SUCCESSES in 2.038 seconds!

Some tests FAILS in 2.039 seconds!

Some tests FAILS in 2.041 seconds!

Some tests FAILS in 2.042 seconds!
==> Testing process ends at 05-Jun-2006 16:51:32

Keep It Simplyer Stupid

This is the simplest usage that can be done with this framework.. and in
general the simplest usage of a unit testing framework: here the current
directory is assumed to be the one containing tests and the
"testrunnerTextSimple" is used as default. Obviously the "cd" commands
can be done simply using the Matlab IDE tools.. so ongly the "testRun"
command must be given (the assignement of the test results can be
omitted.. so only "testRun" ENTER must be written from the user ;).
cd tests;
testsResults = testRun;
cd ..;
==> Testing process starts at 05-Jun-2006 16:51:33
tests
SfactoryTemplate
F
***   ATest: FAILURE! "Comment"
          	lasterr: Output argument "tuo" (and maybe others) not assigned during call to "/home/ilgab/published/matlab/MUnit/tests/SfactoryTemplate.m (MUnitTest_ATest)".
.
Some tests FAILS in 0.002 seconds!
factoryTests
StestTestSet01
.F
***     Cos: FAILURE! "Check the function 'cos'"
          	lasterr: Cos(pi/2) isn't 0!

Some tests FAILS in 0.003 seconds!
anotherTestSet
StestTestSet02
Setting up...
Warning: Matrix is close to singular or badly scaled.
         Results may be inaccurate. RCOND = 1.541976e-18.
F
***    Inv1: FAILURE! "Check the function 'inv'"
          	lasterr: The inverse of the non-singular matrix m1 isn't really the inverse!
Shoutting down...
Setting up...
.Shoutting down...

Some tests FAILS in 0.002 seconds!
StestTestSet03
..
All tests SUCCESSES in 2.038 seconds!

Some tests FAILS in 2.039 seconds!

Some tests FAILS in 2.041 seconds!

Some tests FAILS in 2.042 seconds!
==> Testing process ends at 05-Jun-2006 16:51:37

Contact us