Storing results from unit tests for later review

I have a set of unit test methods defined inside a class derived from matlab.unittest.TestCase. The tests run and give pass/fail results as expected.

What I would like to do is to save a data structure generated during each test, for later review. Currently this will consist of three file names, but I may add to this later.

I have been looking through the unit test framework documentation, but haven't found a way to do this yet. Is it possible to do this?

[Andy Campbell]

Hi Michael, I am not sure yet exactly what you are looking for. Some questions:

• What do you mean by it will consist of three file names?
• Are you looking for a programmatic interface to the test results?
• Are you looking for the test results in a format that is easily archived?
• Are you generating some files or other artifacts during the test run that you would like to save?

I suspect that we may have the capability in the test framework you need, but can you clarify that need so that I am more likely to hit the mark?

[Brendan Hamm]

Also, what is meant by data structure? Do you mean an actual struct?

The result of running as test generates a vector of Test Results, which can easily be converted to a table. Saving this would be as simple as the save command. Is this what you want?

Or, possibly you are interested in logging the output of the tests to a text file. In this case we can create our own matlab.unittest.TestRunner, add a matlab.unittest.plugins.LoggingPlugin to the runner, and call its run method rather than the run method of the TestCase class. Possibly you are interested in another matlab.unittest.plugin as well.

[Michael Lofgren]

Each of my tests use two image files as input, and generates a struct that stores the results of the function under test. At a later time I wish to display those images and overlay data stored in the struct, as a visual way of determining if the test worked at a more detailed level. I can save this data in individual mat files for each test, but that doesn't provide me with a way to programmatically link it with the test and its result.

When runtests() completes, it produces a TestResult array. Within each of those TestResult objects, I would like to include the paths to my image files and the generated struct so that I can view results of each test visually, at a later time. I can then choose to inspect data from all tests, or just from those that have failed, for example.

Hopefully that answers your questions, if not let me know and I'll try to clarify.

[Brendan Hamm]

Sorry for the delay.

I'm a bit confused. If you have the information on the paths of the images created by each test, why can these not be matched up?

Are you using a parameterization of the image paths?

Perhaps a code example may clarify exactly where the issue is.

Answer 1

[Andy Campbell]

Based on the comments above, it sounds like you may want to try an approach simialr to the one outlined in this blog post:

https://blogs.mathworks.com/developer/2018/07/30/semi-automated-testing/

Hope that helps!

Andy

[Michael Lofgren]

Thanks Andy - at a glance your post looks like it might answer my question. I'm working on something else at the moment but will certainly review your post in more detail when I eventually get back to it.

Michael