matlab.perftest.TimeExperiment.withFixedSampleSize

Class: matlab.perftest.TimeExperiment
Package: matlab.perftest

Construct time experiment with fixed number of measurements

Syntax

experiment = matlab.perftest.TimeExperiment.withFixedSampleSize(numSamples)

Description

experiment = matlab.perftest.TimeExperiment.withFixedSampleSize(numSamples) constructs a time experiment with a fixed number of measurements. This method returns an instance of FixedTimeExperiment.

experiment = matlab.perftest.TimeExperiment.withFixedSampleSize(numSamples,'NumWarmups',numWarmups) configures the time experiment to first warm up the code by exercising it numWarmups times.

Input Arguments

numSamples — Number of sample measurements to collect

positive integer

Number of sample measurements to collect, specified as a positive integer. If you specified a number of warm-ups, the testing framework first exercises the code numWarmups times before collecting numSamples measurements.

numWarmups — Number of warm-up measurements

0 (default) | nonnegative integer

Number of warm-up measurements, specified as a nonnegative integer. numWarmups defines the number of times that the test framework exercises the test code to warm it up. Warming up the code gives a more realistic analysis of typical execution time, since it minimizes the effects of first-time run costs.

Example: experiment = matlab.perftest.TimeExperiment.withFixedSampleSize(24,'NumWarmups',8) constructs a FixedTimeExperiment that exercises the code 8 times to warm it up and then exercises the code 24 times to collect sample measurements.

Examples

Performance Test with Fixed Number of Measurements

In your current working folder, create a class-based test, preallocationTest.m, that compares different methods of preallocation.

classdef preallocationTest < matlab.perftest.TestCase
    methods(Test)
        function testOnes(testCase)
            x = ones(1,1e7);
        end
        function testIndexingWithVariable(testCase)
            id = 1:1e7;
            x(id) = 1;
        end
        function testIndexingOnLHS(testCase)
            x(1:1e7) = 1;
        end
        function testForLoop(testCase)
            for i=1:1e7
                x(i) = 1;
            end
        end
        
    end
end

Create a test suite.

suite = testsuite('preallocationTest');

Construct a time experiment with a fixed number of sample measurements, and run the tests.

import matlab.perftest.TimeExperiment
numSamples = 6;
experiment = TimeExperiment.withFixedSampleSize(numSamples);
result = run(experiment,suite);

Running preallocationTest
..........
..........
....
Done preallocationTest
__________

View the test activity for the fourth test.

result(4).TestActivity

ans = 

                Name                 Passed    Failed    Incomplete    MeasuredTime    Objective         Timestamp             Host        Platform           Version                      TestResult                          RunIdentifier            
    _____________________________    ______    ______    __________    ____________    _________    ____________________    ___________    ________    _____________________    ________________________________    ____________________________________

    preallocationTest/testForLoop    true      false     false         0.90553         sample       29-Dec-2015 12:14:55    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf
    preallocationTest/testForLoop    true      false     false         0.86564         sample       29-Dec-2015 12:14:56    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf
    preallocationTest/testForLoop    true      false     false         0.75888         sample       29-Dec-2015 12:14:57    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf
    preallocationTest/testForLoop    true      false     false         0.74051         sample       29-Dec-2015 12:14:58    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf
    preallocationTest/testForLoop    true      false     false          0.8735         sample       29-Dec-2015 12:14:58    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf
    preallocationTest/testForLoop    true      false     false         0.83188         sample       29-Dec-2015 12:14:59    MY-HOSTNAME    win64       9.0.0.320924 (R2016a)    [1x1 matlab.unittest.TestResult]    a07f34c0-5653-4e01-b814-118fe30d3adf

The performance testing framework collected six sample measurements for each test.

Construct a time experiment that also runs the code 3 times to warm it up. Run the tests.

numWarmups = 3;
experiment = TimeExperiment.withFixedSampleSize(numSamples,'NumWarmups',numWarmups);
result = run(experiment,suite);

Running preallocationTest
..........
..........
....
Done preallocationTest
__________

View the test activity for the fourth test.

result(4).TestActivity

ans = 

                Name                 Passed    Failed    Incomplete    MeasuredTime    Objective         Timestamp             Host        Platform           Version                      TestResult                          RunIdentifier            
    _____________________________    ______    ______    __________    ____________    _________    ____________________    ___________    ________    _____________________    ________________________________    ____________________________________

    preallocationTest/testForLoop    true      false     false         0.82972         warmup       29-Dec-2015 12:21:59    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.85917         warmup       29-Dec-2015 12:22:00    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.85857         warmup       29-Dec-2015 12:22:01    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.85307         sample       29-Dec-2015 12:22:02    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.86655         sample       29-Dec-2015 12:22:03    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.81533         sample       29-Dec-2015 12:22:04    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.88266         sample       29-Dec-2015 12:22:04    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false         0.94436         sample       29-Dec-2015 12:22:05    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a
    preallocationTest/testForLoop    true      false     false          1.0375         sample       29-Dec-2015 12:22:07    MY-HOSTNAME    win64       9.0.0.316358 (R2016a)    [1x1 matlab.unittest.TestResult]    37da664a-feba-4277-975f-3d71bcbac71a

For each test, the performance testing framework collected three warm-up measurements in addition to the six sample measurements.