MATLAB xUnit Test Framework

MATLAB xUnit is a unit test framework for MATLAB code.

Description

MATLAB xUnit Test Framework is a unit test framework for MATLAB code.

MATLAB xUnit is designed to be easy to use for MATLAB users with a wide range of experience. Users can write tests using ordinary MATLAB function files that are very simple in structure.

MATLAB xUnit comes with extensive documentation that ranges in scope from a "Getting Started" section to advanced techniques and architectural notes. You can view this documentation online without downloading the package. For example, scroll down to the "Published M Files" section on this page and click on "MATLAB xUnit Quick Start - How to write and run tests." To see all the MATLAB xUnit documentation online, scroll down to the "HTML Files" section on this page and click on "Readme.html."

Only the "xunit" directory is needed to use the framework. The "tests" directory contains the framework's own test suite. The "architecture" directory contains architectural notes on the framework's design and how it might be extended.

MATLAB xUnit has been tested for use with MATLAB R2008a, R2008b, and R2009a. MATLAB xUnit relies heavily on object-oriented language features introduced in R2008a and will not work with earlier releases.

• MATLAB release: MATLAB 7.6 (R2008a)
• Published M Files: How RUNTESTS Searches for Test Cases, How to Put Multiple Test Cases in One M-file, How to Run a Specific Test, How to Run Tests in Specific Directories, How to Run Tests Silently and Query the Results, How to Test an Error Message, How to Test Using a Floating-Point Tolerance, How to Write Tests That Share Common Set-Up Code, How to Write xUnit-Style Tests by Subclassing TestCase, MATLAB xUnit Test Framework: Architectural Notes, Quick Start: How to Write and Run Tests
• HTML Files: MATLAB xUnit Release History,; MATLAB® xUnit Test Framework,; matlab_xunit/doc/index.html,; matlab_xunit/Readme.html

Other Files

license.txt,
matlab_xunit/architecture/class_diagram_a.vsd,
matlab_xunit/architecture/class_diagram_b.vsd,
matlab_xunit/architecture/class_diagram_c.vsd,
matlab_xunit/architecture/html/class_diagram_a.gif,
matlab_xunit/architecture/html/class_diagram_b.gif,
matlab_xunit/architecture/html/class_diagram_c.gif,
matlab_xunit/architecture/matlab_xunit_architecture.m,
matlab_xunit/architecture/testSample.m,
matlab_xunit/doc/example_quick_start/testFliplrMatrix.m,
matlab_xunit/doc/example_quick_start/testFliplrVector.m,
matlab_xunit/doc/example_subfunction_tests/testFliplr.m,
matlab_xunit/doc/examples_general/testBadSinTest.m,
matlab_xunit/doc/examples_general/testCos.m,
matlab_xunit/doc/examples_general/testSetupExample.m,
matlab_xunit/doc/examples_general/testSin.m,
matlab_xunit/doc/examples_general/TestUsingTestCase.m,
matlab_xunit/doc/examples_general/testWithSetupError.m,
matlab_xunit/doc/exException.m,
matlab_xunit/doc/exQuickStart.m,
matlab_xunit/doc/exRunSpecificTest.m,
matlab_xunit/doc/exRunTestsInADirectory.m,
matlab_xunit/doc/exSilentRunning.m,
matlab_xunit/doc/exSubfunctionTests.m,
matlab_xunit/doc/exTestCase.m,
matlab_xunit/doc/exTestCaseSearching.m,
matlab_xunit/doc/exTestFixtures.m,
matlab_xunit/doc/exTolerance.m,
matlab_xunit/doc/file_exchange_description.txt,
matlab_xunit/doc/helptoc.xml,
matlab_xunit/doc/html/exTolerance_eq28823.png,
matlab_xunit/doc/html/exTolerance_eq55725.png,
matlab_xunit/doc/screen-shot.png,
matlab_xunit/obsolete/+mtest/+utils/compareFloats.m,
matlab_xunit/obsolete/+mtest/+utils/containsRegexp.m,
matlab_xunit/obsolete/+mtest/+utils/Contents.m,
matlab_xunit/obsolete/+mtest/+utils/generateDoc.m,
matlab_xunit/obsolete/+mtest/+utils/isAlmostEqual.m,
matlab_xunit/obsolete/+mtest/+utils/isSetUpString.m,
matlab_xunit/obsolete/+mtest/+utils/isTearDownString.m,
matlab_xunit/obsolete/+mtest/+utils/isTestCaseSubclass.m,
matlab_xunit/obsolete/+mtest/+utils/isTestString.m,
matlab_xunit/obsolete/+mtest/+utils/parseFloatAssertInputs.m,
matlab_xunit/obsolete/assertAlmostEqual.m,
matlab_xunit/obsolete/mtest.m,
matlab_xunit/obsolete/tests/cwd_test/TestCaseSubclass.m,
matlab_xunit/obsolete/tests/cwd_test/testFoobar.m,
matlab_xunit/obsolete/tests/cwd_test/testSubfunctions.m,
matlab_xunit/obsolete/tests/MtestTest.m,
matlab_xunit/obsolete/tests/testAssertAlmostEqual.m,
matlab_xunit/obsolete/tests/testIsAlmostEqual.m,
matlab_xunit/tests/almost_black.tif,
matlab_xunit/tests/black.png,
matlab_xunit/tests/black.tif,
matlab_xunit/tests/Contents.m,
matlab_xunit/tests/cwd_test/TestCaseSubclass.m,
matlab_xunit/tests/cwd_test/testFoobar.m,
matlab_xunit/tests/cwd_test/testSubfunctions.m,
matlab_xunit/tests/dir1/test_thatPasses.m,
matlab_xunit/tests/dir2/test_thatFails.m,
matlab_xunit/tests/empty_file,
matlab_xunit/tests/helper_classes/BadFixture.m,
matlab_xunit/tests/helper_classes/Contents.m,
matlab_xunit/tests/helper_classes/ExceptionNotThrownTest.m,
matlab_xunit/tests/helper_classes/FailingTestCase.m,
matlab_xunit/tests/helper_classes/LoggingTestCase.m,
matlab_xunit/tests/helper_classes/NoTestMethods.m,
matlab_xunit/tests/helper_classes/notTestString.m,
matlab_xunit/tests/helper_classes/PassingExceptionTest.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesA.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesB.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesC.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesD.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesE.m,
matlab_xunit/tests/helper_classes/testFunctionHandlesTeardownNoSetup.m,
matlab_xunit/tests/helper_classes/testSimple.m,
matlab_xunit/tests/helper_classes/TestsToBeDiscovered.m,
matlab_xunit/tests/helper_classes/TwoPassingTests.m,
matlab_xunit/tests/helper_classes/WrongExceptionThrownTest.m,
matlab_xunit/tests/Readme.m,
matlab_xunit/tests/RuntestsTest.m,
matlab_xunit/tests/test_arrayToString.m,
matlab_xunit/tests/test_assertElementsAlmostEqual.m,
matlab_xunit/tests/test_assertFilesEqual.m,
matlab_xunit/tests/test_assertVectorsAlmostEqual.m,
matlab_xunit/tests/test_compareFloats.m,
matlab_xunit/tests/test_comparisonMessage.m,
matlab_xunit/tests/test_parseFloatAssertInputs.m,
matlab_xunit/tests/test_stringToCellArray.m,
matlab_xunit/tests/test_TestSuiteInDir.m,
matlab_xunit/tests/testAssertEqual.m,
matlab_xunit/tests/testAssertExceptionThrown.m,
matlab_xunit/tests/testAssertFalse.m,
matlab_xunit/tests/testAssertTrue.m,
matlab_xunit/tests/TestCaseTest.m,
matlab_xunit/tests/TestCaseWithAddPathTest.m,
matlab_xunit/tests/testContainsRegexp.m,
matlab_xunit/tests/TestFuncHandleTests.m,
matlab_xunit/tests/testIsSetUpString.m,
matlab_xunit/tests/testIsTearDownString.m,
matlab_xunit/tests/testIsTestCaseSubclass.m,
matlab_xunit/tests/testIsTestString.m,
matlab_xunit/tests/TestRunLoggerTest.m,
matlab_xunit/tests/testRuntestsWithDirectoryName.m,
matlab_xunit/tests/TestSuiteTest.m,
matlab_xunit/tests/ThrowsExceptionTest.m,
matlab_xunit/xunit/+xunit/+utils/arrayToString.m,
matlab_xunit/xunit/+xunit/+utils/compareFloats.m,
matlab_xunit/xunit/+xunit/+utils/comparisonMessage.m,
matlab_xunit/xunit/+xunit/+utils/containsRegexp.m,
matlab_xunit/xunit/+xunit/+utils/Contents.m,
matlab_xunit/xunit/+xunit/+utils/generateDoc.m,
matlab_xunit/xunit/+xunit/+utils/isAlmostEqual.m,
matlab_xunit/xunit/+xunit/+utils/isSetUpString.m,
matlab_xunit/xunit/+xunit/+utils/isTearDownString.m,
matlab_xunit/xunit/+xunit/+utils/isTestCaseSubclass.m,
matlab_xunit/xunit/+xunit/+utils/isTestString.m,
matlab_xunit/xunit/+xunit/+utils/parseFloatAssertInputs.m,
matlab_xunit/xunit/+xunit/+utils/stringToCellArray.m,
matlab_xunit/xunit/assertElementsAlmostEqual.m,
matlab_xunit/xunit/assertEqual.m,
matlab_xunit/xunit/assertExceptionThrown.m,
matlab_xunit/xunit/assertFalse.m,
matlab_xunit/xunit/assertFilesEqual.m,
matlab_xunit/xunit/assertTrue.m,
matlab_xunit/xunit/assertVectorsAlmostEqual.m,
matlab_xunit/xunit/CommandWindowTestRunDisplay.m,
matlab_xunit/xunit/Contents.m,
matlab_xunit/xunit/FunctionHandleTestCase.m,
matlab_xunit/xunit/info.xml,
matlab_xunit/xunit/initTestSuite.m,
matlab_xunit/xunit/runtests.m,
matlab_xunit/xunit/TestCase.m,
matlab_xunit/xunit/TestCaseInDir.m,
matlab_xunit/xunit/TestCaseWithAddPath.m,
matlab_xunit/xunit/TestComponent.m,
matlab_xunit/xunit/TestComponentInDir.m,
matlab_xunit/xunit/TestRunLogger.m,
matlab_xunit/xunit/TestRunMonitor.m,
matlab_xunit/xunit/TestSuite.m,
matlab_xunit/xunit/TestSuiteInDir.m

[Ryan Ollos]

 

[Rick Bosi]

What can I say... Thank you for building and providing this. This should be part of the base MATLAB package!

[Shi vera]

That's exactly what i need! Thank you very much!

[gg]

Steve, thank you for MTEST available I find it very useful.

Nevertheless I encountered a problem involving the mtest help-file. Choosing "MTEST Unit

Test Framework" in the help window causes Matlab to freeze. No input (mouse and keyboard) is

accepted and the application needs to be terminated by the task-manager. This problem can be

replicated by different users on different machines.

Details to replicate the problem:

- Copy Mtest to <mtest-dir>
- "Add Path with Subfolders" <mtest-dir>
- Open help in Matlab
- click on "MTEST Unit Test Framework"
=> Freeze

All other mtest help entries work as expected.

Matlab version: R2009a, 7.8.0.324 (win32)
OS: Windows XP Professional, 5.1.2600 SP3

It is really more a nuisance than a no-go but maybe you can track down the problem.

Thank you & kind regards
Guido

[Steve Eddins]

Hi Guido,

I have not been able to reproduce the problem using R2009a on my 32-bit WinXP SP3 machine.

Would you please double-check your MATLAB version? I believe the released version was 7.8.0.347, not 7.8.0.324 as you reported.

Thanks,

Steve

[Steve Eddins]

Hi again Guido,

The Help Browser team was able to reproduce the problem you reported, and they are investigating it. I don't know yet if there's a work-around I could apply to the MTEST doc files.

Steve

[Ryan Ollos]

This is very nicely written, and useful. For those MATLAB coders that desire to learn better practices.

[Ryan Ollos]

I'm also having the issue with the doc browser freezing, but I find it doesn't occur if I type 'doc mtest' from the command line to access the help, rather than trying to expand the entry in the help browser.

>> version
ans =
7.8.0.347 (R2009a)

One small feature request would be to allow messages with sprintf style conversion specifiers.

[Steve Eddins]

Ryan and Guido, the Help Browser team has identified the problem and is trying to fix the problem for the next MATLAB release. In the meantime, you can try this workaround: Put the MTEST directory after the MATLAB directories in your path, instead of before them.

Ryan, thanks for the enhancement suggestion. I have received several good suggestions, and I plan to work on an MTEST update soon.

[Ming Li]

Thank you for putting this together and provide support for the framework. Great work.

[Steve Eddins]

Ryan, I decided not to do the sprintf-style conversion specifiers in the assert functions, as least for now. My experience designing syntaxes for MATLAB and toolbox functions makes me shy away from a syntax design with an indefinite set of trailing arguments, as in sprintf. Such a syntax design makes it difficult to expand the syntax to support other options in the future. For now, you can just inline a call to sprintf directly in the call to the assertion functions.

[Frank]

Steve, thanks for making this available. I had used xtargets/munit in the past, and while quite powerful, the syntax was different from other *Unit frameworks, which I felt took away from it a little. I've used *Unit in Python, Perl, Java, and C - I'm used to the way they work, I didn't really want to learn a "better" way to do it, much less teach my colleagues to use it. I'm glad to have the familiar framework available now in MATLAB.

As you mentioned in your 2006 talk (I just read the slides), most of us are not primarily programmers - we're scientists, engineers, etc. However, we use languages like Python and MATLAB to express our scientific ideas. I've long been a believer in using software engineering tools to make my *science* better, easier, more reliable and more reproduceable, and I agree that unit-testing is one of the keys to that. Still (as you also point out), it's not widely appreciated in the scientific community, and no scientist ever formally learns about these tools.

Having a simple interface to unit-testing in MATLAB is a big win, in my book. It should make convincing my colleagues to use it a whole easier. Thanks

[Nick]

Great software! Thanks. I noticed a bug:

function testStuff
    assertElementsAlmostEqual(0,Inf)
    assertVectorsAlmostEqual(0,Inf)
end

Will not flag a failure.

--N

[Nick]

I think there is a divide by inf in computing the relative tolerance. Using the 'absolute' flag provides the correct result.

[Nick]

If you are putting multiple tests in a single file, don't start any function names with "setup". The following code fails. Remove setup and it works.

function test_suite = testFliplr
initTestSuite;
end

function testFliplrMatrix
in = magic(3);
assertEqual(fliplr(in), in(:, [3 2 1]));
end

function testFliplrVector
assertEqual(fliplr([1 4 10]), [10 4 1]);
end

function test3
[a b] = setupdata;
assertEqual(a,1);
assertEqual(b,2);
end

function [a b] = setupdata
a = 1;
b = 2;
end

[Steve Eddins]

Nick, the behavior with your setupdata function is a feature. See the doc page called "How to Write Tests That Share Common Set-Up Code."

http://www.mathworks.com/matlabcentral/fx_files/22846/6/content/matlab_xunit/doc/html/exTestFixtures.html

[Steve Eddins]

Nick, thanks for the error report about assertElementsAlmostEqual and assertVectorsAlmostEqual. I uploaded a new version (2.0.1) today that fixes the problem.

[Jader]

I couldn't make it work on Matlab R2008b 7.7 and Windows 7
It doesn't finds the m-files with test functions in the current pwd

>> runtests
Starting test run with 0 test cases.

PASSED in 0.000 seconds.

[Jader]

I found the source of my problem: My test files names did not begun with "Test"